Docs update

colvert-project · Jan 27, 2025 · 5bab34c · 5bab34c
1 parent 3862d2b
commit 5bab34c
Show file tree

Hide file tree

Showing 6 changed files with 153 additions and 77 deletions.
diff --git a/docs/Connectors.md b/docs/Connectors.md
@@ -0,0 +1,5 @@
+# Connectors
+
+This part contains documentation from installed connectors.
+
+TODO
diff --git a/docs/Dev-General.md b/docs/Dev-General.md
@@ -0,0 +1,46 @@
+# General
+
+This part contains information intended to help developers to understand the inner workings of Colvert and connectors.
+
+This **Developer Documentation** may be refered as **"CodeDoc"**.
+
+## Development Environment
+
+### GitHub Codespaces Setup
+
+Create a personal **GitHub Codespaces** from scratch with ```colvert-project/colvert``` and ```origin/main``` remote repository.
+
+Some Visual Studio Code extensions can be settled in addition of default ones:
+
+* Python (_Microsoft_)
+* Python Debugger (_Microsoft_)
+* Pylance (_Microsoft_)
+* GitHub Actions (_GitHub_)
+* GitHub Copilot (_GitHub_)
+* GitHub Copilot Chat (_GitHub_)
+* markdownlint (_David Anson_)
+* Docker (_Microsoft_)
+* SQLite Viewer (_Florian Klampfer_)
+* autoDocstring - Python Docstring Generator (_Nils Werner_)
+    + <https://sphinx-rtd-tutorial.readthedocs.io/en/latest/docstrings.html>
+    + Settings > Extensions > Python Docstring Generator configuration > _Docstring Format_ > `sphinx`
+
+As reminder, in **GitHub Codespaces** states concerning opened tabs and settings (like color theme, etc.) are not saved until you set _Backup and Sync Settings_.
+
+## Documentation Build
+
+It uses **Sphinx** with a third-party GitHub workflow in `.github/workflows/build-docs.yml` to trigger build on `git push` action. It uses the third-party action [`sphinx-notes/pages@v3`](https://github.com/sphinx-notes/pages).
+
+|File                   |Usage                                                                                                                    |
+|-----------------------|-------------------------------------------------------------------------------------------------------------------------|
+|`static/`              |Classic favicon, logo, and other static file to serve.                                                                   |
+|`docs/requirements.txt`|pip packages needed by Sphinx to build documentation. Commented.                                                         |
+|`docs/conf.py`         |Sphinx configuration. Commented.                                                                                         |
+|`CNAME`                |Define the base URL for documentation. Not used by GitHub as built using a workflow, but used by `conf.py`.              |
+|`index.rst.j2`         |Jinja2 parsed first template (`conf.py`) to load docs from connectors, then Sphinx index using _reStructuredText_ format.|
+|`[*/]*.md`             |All others documentation files in Markdown format.                                                                       |
+
+Help links:
+
+* Sphinx docs for `conf.py`: <https://www.sphinx-doc.org/en/master/usage/configuration.html>
+* As we use `sphinx_rtd_theme` for Sphinx, here is some help for **docstrings** writing: <https://sphinx-rtd-tutorial.readthedocs.io/en/latest/docstrings.html>
diff --git a/docs/Getting-Started.md b/docs/Getting-Started.md
@@ -1,20 +1,24 @@
-# Getting started
+# Getting Started
+
+General information available in [README.md](https://github.com/colvert-project/colvert/blob/main/README.md).
 
 ## Requirements
 
 TODO
 
-## Installing Colvert
+## Deploy Colvert
 
 ### Git Clone
 
-TODO
+```shell
+git clone https://github.com/colvert-project/colvert.git
+```
 
 ### PyPI
 
 TODO
 
-### Docker
+### PaaS
 
 #### Docker Hub
 
@@ -28,4 +32,10 @@ TODO
 
 TODO
 
+#### Dokploy
+
+TODO
+
 ### Offline
+
+TODO
diff --git a/docs/General.md → docs/Software-Architecture.md b/docs/General.md → docs/Software-Architecture.md
@@ -1,42 +1,4 @@
-# General
-
-This **Developer Documentation** may be refered as **"CodeDoc"**.
-
-## Development Environment
-
-### GitHub Codespaces
-
-Create a personal **GitHub Codespaces** from scratch with ```colvert-project/colvert``` and ```origin/main``` remote repository.
-
-Some Visual Studio Code extensions can be settled in addition of default ones:
-
-* Python (_Microsoft_)
-* Python Debugger (_Microsoft_)
-* Pylance (_Microsoft_)
-* GitHub Actions (_GitHub_)
-* markdownlint (_David Anson_)
-* Docker (_Microsoft_)
-
-As reminder, in **GitHub Codespaces** states concerning opened tabs and settings (like color theme, etc.) are not saved until you set _Backup and Sync Settings_.
-
-## Documentation Build
-
-It uses **Sphinx** with a third-party GitHub workflow in `.github/workflows/build-docs.yml` to trigger build on `git push` action. It uses the third-party action [`sphinx-notes/pages@v3`](https://github.com/sphinx-notes/pages).
-
-|File                   |Usage                                                                                                                    |
-|-----------------------|-------------------------------------------------------------------------------------------------------------------------|
-|`static/`              |Classic favicon, logo, and other static file to serve.                                                                   |
-|`docs/requirements.txt`|pip packages needed by Sphinx to build documentation. Commented.                                                         |
-|`docs/conf.py`         |Sphinx configuration. Commented.                                                                                         |
-|`CNAME`                |Define the base URL for documentation. Not used by GitHub as built using a workflow, but used by `conf.py`.              |
-|`index.rst.j2`         |Jinja2 parsed first template (`conf.py`) to load docs from connectors, then Sphinx index using _reStructuredText_ format.|
-|`*.md`                 |All others documentation files in Markdown format.                                                                       |
-
-Help links:
-
-* Sphinx docs for `conf.py`: <https://www.sphinx-doc.org/en/master/usage/configuration.html>
-
-# Architecture
+# Software Architecture
 
 **Colvert** is a web application written in **_Python_ 3**, built on **_Django_ 5** web framework with **_AdminLTE 4_** templates library for view and control parts, itself based on **_Bootstrap_ 5**.
 
@@ -45,7 +7,7 @@ Help links:
 |Library|Type|Version|Purpose|
 |----------|------|------|-|
 |_Python_      |Engine|3.12.1     |Language in which Colvert is written with|
-|**Django**    |Python|5.1.1      |Web Application Framework (Basic application features, Templating, ORM)|
+|**Django**    |Python|5.1.5      |Web Application Framework (Basic application features, Templating, ORM)|
 |**WhiteNoise**|Python|6.7.0      |Server properly static files in production **AND** in debug mode|
 |AdminLTE      |CSS+JS|4.0.0-beta2|Front-end library used with HTML templates|
 |Bootstrap     |CSS+JS|5().3?       ||
@@ -96,6 +58,14 @@ pip 24.2 from /usr/local/python/3.12.1/lib/python3.12/site-packages/pip (python
 
 * Applying Quickstart (only for MIDDLEWARE), 1., 2. & 5.
 
+#### Python Libraries Upgrade
+
+* pip
+
+```shell
+python -m pip install --upgrade pip
+```
+
 ### Django Project Initialization
 
 Within the root repository folder ```/workspaces/colvert```.
@@ -107,7 +77,7 @@ Within the root repository folder ```/workspaces/colvert```.
 
 Hint: <https://automationpanda.com/2018/02/06/starting-a-django-project-in-an-existing-directory/>
 
-## Additional Libraries Installation
+### Additional Libraries Installation
 
 ```shell
 /workspaces/colvert (main) $ npm --version
@@ -116,6 +86,7 @@ Hint: <https://automationpanda.com/2018/02/06/starting-a-django-project-in-an-ex
 
 * **AdminLTE**, admin dashboard & control panel theme. Built on top of Bootstrap.
 
+References:
 <https://adminlte.io/>  
 <https://adminlte.io/docs/>
 
@@ -141,17 +112,63 @@ For future library update, run again commands above by replacing the Bootstrap /
 
 **Docstrings** is done using _reStructuredText_ formatting.
 
-### Packaging & Run
+### Model Management
 
-Update `requirements.txt`
+#### Strategy
+
+We avoid :
+
+```shell
+# Generate an empty migration file from the given app
+python manage.py makemigrations --empty core --name add_defaults
+```
+
+TODO doc. signal
+
+#### Reset migration files & DB
+
+1. Delete `migrations/__pycache__`.
+2. Delete problematic or all `migrations/<XXXX_MIG>.py` files.
+3. Delete `db.sqlite3` file.
+4. `python manage.py makemigrations` (create also empty db.sqlite3 if not existing).
+5. `python manage.py migrate` to apply model in DB, will also set default values if they do not exist using the Django Signals approach.
+
+#### Model Migration Cheatsheet
+
+```shell
+# Generate migration files from the given app, or for all apps if not given
+python manage.py makemigrations [<APP>]
+
+# TODO
+python manage.py squashmigrations core 0001 0003
+
+# To apply model in DB
+python manage.py migrate
+
+# TODO
+python manage.py showmigrations
+```
+
+### Fixtures
+
+TODO: python manage.py loaddata core/fixtures
+
+#### Static Files Management
 
 ```shell
-python manage.py migrate # To update model if changed
 python manage.py collectstatic # To publish all static files in STATIC_ROOT if new ones need to
-python manage.py runserver # Start development server
 ```
 
-TODO - Merge migrations
+### Packaging & Run
+
+1. Update `requirements.txt`
+2. Migrate model if needed
+3. Publish static files if needed
+4. Run development server:
+
+```shell
+python manage.py runserver
+```
 
 #### References
 

diff --git a/docs/connectors/Mitre-Attack.md b/docs/connectors/Mitre-Attack.md
@@ -0,0 +1,3 @@
+# Mitre Attack
+
+TODO
diff --git a/docs/index.rst.j2 b/docs/index.rst.j2
@@ -12,71 +12,66 @@ The tool consists of:
 * The main application.
 * Connectors to potentially get data from providers and contextual more-valued information.
 
-Go to the main repository <https://github.com/colvert-project/colvert> for an overview of the tool.
+Go to the main repository https://github.com/colvert-project/colvert for an overview of the tool.
 
-* Website: <https://colvert.io>
+* Website: https://colvert.io
 * Contact: [email protected]
 
 Interesting Places
 ==================
 
-* **Last Release:** <https://github.com/colvert-project/colvert/releases/latest>
-* **Packages:** <https://github.com/orgs/colvert-project/packages>
-* **Discussions:** <https://github.com/orgs/colvert-project/discussions>
-* **Issues Tracker:** <https://github.com/colvert-project/colvert/issues>
+* **Packages:** https://github.com/orgs/colvert-project/packages
+* **Discussions:** https://github.com/orgs/colvert-project/discussions
+* **Issues Tracker:** https://github.com/colvert-project/colvert/issues
 
-* **Security Policy:** <https://github.com/colvert-project/colvert/security/policy>
-* **Security Advisories:** <https://github.com/colvert-project/colvert/security/advisories>
+* **Security Policy:** https://github.com/colvert-project/colvert/security/policy
+* **Security Advisories:** https://github.com/colvert-project/colvert/security/advisories
 
 Contributing
 ============
 
-* How to contribute?
-   * <https://github.com/colvert-project/colvert/blob/main/CONTRIBUTING.md>
+* How to contribute? 
+https://github.com/colvert-project/colvert/blob/main/CONTRIBUTING.md
 
-* Colvert Contributor Code of Conduct:
-   * <https://github.com/colvert-project/colvert/blob/main/CODE_OF_CONDUCT.md>
+* Colvert Contributor Code of Conduct: 
+https://github.com/colvert-project/colvert/blob/main/CODE_OF_CONDUCT.md
+
+:tocdepth: 3
 
 .. toctree::
-   :maxdepth: 3
    :caption: Usage Guide
 
    Getting-Started.md
 
-The following section contains documentation from installed connectors.
-
 .. toctree::
-   :maxdepth: 3
    :caption: Connectors Documentation
-{% for connector_doc in connector_docs %}
-   {{ connector_doc }}
-{% endfor %}
 
-The following section contains information intended to help developers to understand the inner workings of the Colvert tool and Colvert connectors.
+   Connectors.md
+   {% for connector in connectors %}
+      {{ connector }}
+   {% endfor %}
 
 .. toctree::
-   :maxdepth: 2
    :caption: Developer Documentation
 
-   General.md
+   Dev-General.md
+   Software-Architecture.md
    How-to-Build-Connectors.md
    Credits.md
 
 .. toctree::
-   :maxdepth: 3
    :caption: Core System Docstrings
 
    _generated/modules
 
 .. toctree::
-   :maxdepth: 1
    :caption: About
 
    Changelog <https://github.com/colvert-project/colvert/releases>
+   colvert.io <https://colvert.io>
 
-Indices and tables
-==================
+Indices
+=======
 
 * :ref:`genindex`
 * :ref:`modindex`
-* :ref:`search`