feat(docs): add architecture/roadmap

.pre-commit-config.yaml

@@ -15,6 +15,7 @@ repos:
     -   id: trailing-whitespace
     -   id: end-of-file-fixer
     -   id: check-yaml
+        exclude: mkdocs.yml
     -   id: mixed-line-ending
 -   repo: https://github.com/astral-sh/ruff-pre-commit
     rev: v0.11.13

docs/docs/develop/architecture.md

@@ -0,0 +1,81 @@
+---
+title: Architecture
+---
+
+## Typical Deployment
+
+InvenTree is a classical Django Application and supports the WSGI interface standard. The following diagram shows a typical and recommended deployment architecture of InvenTree.
+
+``` mermaid
+flowchart LR
+    Request --- RP[1: Reverse Proxy]
+    RP --- Gunicorn[5: Gunicorn]
+    subgraph image:inventree/inventree
+    Gunicorn --- Django[6: Django Server processes]
+    end
+    Django --- Database@{ shape: cyl, label: "SQL Database" }
+    Django --- Redis
+    Worker[7: Q2 Worker] ---  Redis@{ shape: cyl, label: "9: Redis Cache" }
+      Database -- 8: DB holds tasks ---> Worker
+
+    subgraph caddy
+    RP --- file[2: file server]
+end
+
+    file --- s[3: Static Files]
+    file --- m[Media Files]
+    RP-. 4: API auth request .-> Gunicorn
+```
+
+1. A Request is received by a reverse proxy (e.g. Caddy, Nginx, Apache).
+2. Requests for static or media files are either served directly by the reverse proxy or forwarded to a dedicated file server
+3. Static or media files can be served by the different file server (placing static files in a CDN for example)
+4. Media files require an API authentication request to the Django server to ensure the user has access to the file
+5. API or frontend requests are forwarded from the reverse proxy to Gunicorn, which is a WSGI server that handles server Django processes.
+6. The Gunicorn processes are loosely coubled instances of Django application - which mainly serves a REST API as the frontend only needs a few full django calls (see [frontend architecture](#frontend-architecture) below).
+7. A properly configured InvenTree instance will also run background workers (Q2 Worker) which are responsible for processing long-running tasks, such as sending notifications, generating reports or calculating expensive updates. Q2 is used as the task processing library, which runs multiple loosely coupled worker processes.
+8. The database holds tasks and is queried by the workers. This enables relatively durable task processing, as the underlying server can be restarted with minimal task loss.
+9. Various components of InvenTree can benefit from a Redis or protocol compatible cache, which is used to store frequently accessed data, such as user sessions, API tokens, global or plugin settings, or other transient data. This can help to improve performance and reduce load on the database.
+
+## Code Architecture
+
+This sections describes various architectural aspects of the InvenTree codebase and mechanisms and lifecycles in the system.
+
+Knowledege of these aspects is not required to use InvenTree, but it is helpful for developers who want to contribute to the project or to understand where / how one might extend the system with plugins or by pathching in custom functionality.
+
+### Repository layout and separation of concerns
+
+All code that is intended to be executed on the server is located in the `src/` directory. Some code in contrib might be needed to deploy or maintain an instance.
+One exception is the `tasks.py` file, that contains definitions for various maintenance tasks that can be run from the command line. This file is located in the root directory of the repository to make instructions easier.
+
+Code styles are generally only applied to the code in the `src/` directory.
+
+### Backend Architecture
+
+InvenTree's backend is a Django application. It is generally structured in a way that follows Django's conventions, but also includes some customizations and extensions to support the specific needs of InvenTree.
+Most remarkable deviations from the Django standard are:
+- Manipulation of the django app mechanisms to enable the [plugin system](#plugin-system)
+- Usage of a custom role-mapping system to make permissions more approachable
+
+The backend aims to be:
+- API first, with a RESTful API that is used by the frontend and can also be used by other applications.
+- Modular, with a clear separation of concerns between different components and apps.
+- Tested reasonably throughout with transparent test coverage
+- Following the Django and generally accepted security conventions
+
+### Frontend Architecture
+
+InvenTree's frontend is primarily a single-page application (SPA) built with React, mantine and yarn/vite for bundling.
+
+#### Serving the frontend
+It is statically pre-build and is served by the Django application as a bundle of static files. No node executable or bundling technology is required to run the frontend in production. It is however possible to use the development server for the frontend stack together with a development or production instance of the backend.
+
+Frontend and backend do not need to be serverd by the same server or on the same domain, only the API versions need to match.
+
+#### Coupling to the backend
+
+The frontend is not coupled during the lifetime of the application but it can be primed via the `INVENTREE_SETTINGS` global variable (rendered by the `spa_settings` tag).
+
+These settings can configure the base url, available backends, environment details and more.
+
+To facilitate this the html structure for the frontend is run through a performance-optimized template. There is only a very limited amount of rendering done to keep response times and surface for  security threads low.

docs/docs/develop/contributing.md

@@ -246,6 +246,12 @@ T002: Double quotes should be used in tags
 
 New features or updates to existing features should be accompanied by user documentation.
 
+### Stable link references
+
+The documentation framework enables addition of redirections. This is used to build stable references for linking in external resources.
+
+New references can be added in `docs/mkdocs.yml` in the `redirect_maps` section. Both external targets and documentation pages are possible targets. All references are linted in the docs CI pipeline.
+
 ## Translations
 
 Any user-facing strings *must* be passed through the translation engine.

docs/docs/develop/roadmap.md

@@ -0,0 +1,35 @@
+---
+title: High level roadmap
+---
+
+## General goals
+
+## High level Epics
+
+### 1.0
+
+Smaller items can be viewed [on the milestone](https://github.com/inventree/InvenTree/issues?q=is%3Aissue%20milestone%3A1.0.0).
+
+Aiming to stabelise several aspects of the software:
+
+- Only shipping the new frontend and removing reliance on templating
+- Stabelize API for client generation
+- Making data import- / export-mechanisms more stable
+- Updating and Re-Organising documentation to enable CII best practices compliance
+
+### 2.0
+
+Smaller items can be viewed [on the milestone](https://github.com/inventree/InvenTree/issues?q=is%3Aissue%20milestone%3A2.0.0).
+
+*Proposed* goals:
+
+- Reorganise permission system to support more entrerprise structures and reduce unneeded permissions [EPIC](https://github.com/inventree/InvenTree/issues/7466)
+- Add generalised file handling [EPIC](https://github.com/inventree/InvenTree/issues/5703)
+
+### Future
+
+There are several epics that target [the horizion](https://github.com/inventree/InvenTree/issues?q=is%3Aissue%20state%3Aopen%20type%3AEpic).
+
+## Non-Goals
+
+TBD

docs/docs/plugins/mixins/export.md

@@ -24,7 +24,7 @@ Each plugin can dictate which datasets are supported using the `supports_export`
       summary: False
       members: []
       extra:
-        show_sources: True
+        show_source: True
 
 The default implementation returns `True` for all data types.
 
@@ -40,7 +40,7 @@ The `generate_filename` method constructs a filename for the exported file.
       summary: False
       members: []
       extra:
-        show_sources: True
+        show_source: True
 
 ### Adjust Columns
 
@@ -54,7 +54,7 @@ The `update_headers` method allows the plugin to adjust the columns selected to
       summary: False
       members: []
       extra:
-        show_sources: True
+        show_source: True
 
 ### Queryset Filtering
 
@@ -68,7 +68,7 @@ The `filter_queryset` method allows the plugin to provide custom filtering to th
       summary: False
       members: []
       extra:
-        show_sources: True
+        show_source: True
 
 ### Export Data
 
@@ -82,7 +82,7 @@ The `export_data` method performs the step of transforming a [Django QuerySet]({
       summary: False
       members: []
       extra:
-        show_sources: True
+        show_source: True
 
 Note that the default implementation simply uses the builtin tabulation functionality of the provided serializer class. In most cases, this will be sufficient.
 

docs/docs/plugins/mixins/ui.md

@@ -24,7 +24,7 @@ The entrypoint for user interface plugins is the `UserInterfaceMixin` class, whi
       summary: False
       members: []
       extra:
-        show_sources: True
+        show_source: True
 
 Note here that the `get_ui_features` calls other methods to extract the available features from the plugin, based on the requested feature type. These methods can be overridden to provide custom functionality.
 
@@ -43,7 +43,7 @@ The `get_ui_features` method should return a list of `UIFeature` objects, which
       summary: False
       members: []
       extra:
-        show_sources: True
+        show_source: True
 
 Note that the *options* field contains fields which may be specific to a particular feature type - read the documentation below on each feature type for more information.
 
@@ -79,7 +79,7 @@ The InvenTree dashboard is a collection of "items" which are displayed on the ma
       show_root_heading: False
       show_root_toc_entry: False
       extra:
-        show_sources: True
+        show_source: True
       summary: False
       members: []
 
@@ -93,7 +93,7 @@ The *options* field in the returned `UIFeature` object can contain the following
       show_root_heading: False
       show_root_toc_entry: False
       extra:
-        show_sources: True
+        show_source: True
       summary: False
       members: []
 
@@ -115,7 +115,7 @@ Many of the pages in the InvenTree web interface are built using a series of "pa
       show_root_heading: False
       show_root_toc_entry: False
       extra:
-        show_sources: True
+        show_source: True
       summary: False
       members: []
 
@@ -129,7 +129,7 @@ The *options* field in the returned `UIFeature` object can contain the following
       show_root_heading: False
       show_root_toc_entry: False
       extra:
-        show_sources: True
+        show_source: True
       summary: False
       members: []
 
@@ -151,7 +151,7 @@ The `get_ui_template_editors` feature type can be used to provide custom templat
       show_root_heading: False
       show_root_toc_entry: False
       extra:
-        show_sources: True
+        show_source: True
       summary: False
       members: []
 
@@ -165,7 +165,7 @@ The `get_ui_template_previews` feature type can be used to provide custom templa
       show_root_heading: False
       show_root_toc_entry: False
       extra:
-        show_sources: True
+        show_source: True
       summary: False
       members: []
 

docs/docs/plugins/mixins/validation.md

@@ -26,7 +26,7 @@ A custom plugin may implement the `validate_model_deletion` method to perform cu
       show_root_heading: False
       show_root_toc_entry: False
       extra:
-        show_sources: True
+        show_source: True
       summary: False
       members: []
 
@@ -42,7 +42,7 @@ Any plugin which inherits the `ValidationMixin` can implement the `validate_mode
       show_root_heading: False
       show_root_toc_entry: False
       extra:
-        show_sources: True
+        show_source: True
       summary: False
       members: []
 
@@ -116,7 +116,7 @@ If the custom method determines that the part name is *objectionable*, it should
       show_root_heading: False
       show_root_toc_entry: False
       extra:
-        show_sources: True
+        show_source: True
       summary: False
       members: []
 
@@ -130,7 +130,7 @@ Validation of the Part IPN (Internal Part Number) field is exposed to custom plu
       show_root_heading: False
       show_root_toc_entry: False
       extra:
-        show_sources: True
+        show_source: True
       summary: False
       members: []
 
@@ -144,7 +144,7 @@ Validation of the Part IPN (Internal Part Number) field is exposed to custom plu
       show_root_heading: False
       show_root_toc_entry: False
       extra:
-        show_sources: True
+        show_source: True
       summary: False
       members: []
 
@@ -162,7 +162,7 @@ The `validate_batch_code` method allows plugins to raise an error if a batch cod
       show_root_heading: False
       show_root_toc_entry: False
       extra:
-        show_sources: True
+        show_source: True
       summary: False
       members: []
 
@@ -176,7 +176,7 @@ The `generate_batch_code` method can be implemented to generate a new batch code
       show_root_heading: False
       show_root_toc_entry: False
       extra:
-        show_sources: True
+        show_source: True
       summary: False
       members: []
 
@@ -196,7 +196,7 @@ Custom serial number validation can be implemented using the `validate_serial_nu
       show_root_heading: False
       show_root_toc_entry: False
       extra:
-        show_sources: True
+        show_source: True
       summary: False
       members: []
 
@@ -236,7 +236,7 @@ A custom plugin can implement the `convert_serial_to_int` method to determine ho
       show_root_heading: False
       show_root_toc_entry: False
       extra:
-        show_sources: True
+        show_source: True
       summary: False
       members: []
 
@@ -255,7 +255,7 @@ For custom serial number schemes, it is important to provide a method to generat
       show_root_heading: False
       show_root_toc_entry: False
       extra:
-        show_sources: True
+        show_source: True
       summary: False
       members: []
 

docs/main.py

@@ -81,7 +81,7 @@ def get_repo_url(raw=False):
     mkdocs_yml = here.joinpath('mkdocs.yml')
 
     with open(mkdocs_yml, encoding='utf-8') as f:
-        mkdocs_config = yaml.safe_load(f)
+        mkdocs_config = yaml.load(f, yaml.BaseLoader)
         repo_name = mkdocs_config['repo_name']
 
     if raw:

docs/mkdocs.yml

@@ -91,6 +91,8 @@ nav:
         - Custom Barcodes: barcodes/custom.md
     - Development:
       - Contributing: develop/contributing.md
+      - Architecture: develop/architecture.md
+      - Roadmap: develop/roadmap.md
       - Devcontainer: develop/devcontainer.md
       - React Frontend: develop/react-frontend.md
     - Mobile App:
@@ -260,6 +262,9 @@ plugins:
       opening_tag: "{!"
       closing_tag: "!}"
   - search
+  - mermaid2:
+      # version: 11.6.0
+      javascript: javascripts/mermaid.min.js
   - git-revision-date-localized
   - mkdocs-simple-hooks:
       hooks:
@@ -285,6 +290,8 @@ plugins:
         'sref/contrib.md': 'develop/contributing.md'  #  https://github.com/inventree/InvenTree/blob/master/CONTRIBUTING.md.
         'sref/docs.md': 'index.md' #   https://docs.inventree.org/en/latest/
         'sref/api.md': 'api/index.md' #  https://demo.inventree.org/api-doc/
+        'sref/architecture.md': 'develop/architecture.md'
+        'sref/roadmap.md': 'develop/roadmap.md'
         'sref/code.md': 'https://github.com/inventree/InvenTree/tree/master' #  https://github.com/inventree/InvenTree/tree/master
         'sref/oci-image.md': 'https://hub.docker.com/r/inventree/inventree/tags' #  https://hub.docker.com/r/inventree/inventree/tags
         'sref/releases.md': 'https://github.com/inventree/InvenTree/releases' # https://github.com/inventree/InvenTree/releases
@@ -311,7 +318,7 @@ markdown_extensions:
       custom_fences:
         - name: mermaid
           class: mermaid
-          # format: !!python/name:pymdownx.superfences.fence_code_format
+          format: !!python/name:mermaid2.fence_mermaid_custom
   # - pymdownx.emoji:
   #     emoji_index: !!python/name:materialx.emoji.twemoji
   #     emoji_generator: !!python/name:materialx.emoji.to_svg

docs/requirements.in

@@ -7,3 +7,4 @@ mkdocs-simple-hooks>=0.1,<1.0
 mkdocs-include-markdown-plugin
 neoteroi-mkdocs
 mkdocstrings[python]>=0.25.0,<=0.29.1
+mkdocs-mermaid2-plugin