Plugins v1

CHANGELOG.md

@@ -4,6 +4,8 @@ Note: Breaking changes between versions are indicated by "💥".
 
 ## Unreleased
 
+- 💥[Improvement] Complete overhaul of the plugin extension mechanism. Tutor now has a hook-based Python API: actions can be triggered at different points of the application life cycle and data can be modified thanks to custom filters. The v0 plugin API is still supported, for backward compatibility, but plugin developers are encouraged to migrate their plugins to the new API. See the new plugin tutorial for more information.
+- [Improvement] Improved the output of `tutor plugins list`.
 - [Feature] Add `tutor [dev|local|k8s] status` command, which provides basic information about the platform's status.
 - 💥[Improvement] Make it possible to run `tutor k8s exec <command with multiple arguments>` (#636). As a consequence, it is no longer possible to run quoted commands: `tutor k8s exec "<some command>"`. Instead, you should remove the quotes: `tutor k8s exec <some command>`.
 

Makefile

@@ -28,17 +28,19 @@ push-pythonpackage: ## Push python package to pypi
 
 test: test-lint test-unit test-types test-format test-pythonpackage ## Run all tests by decreasing order of priority
 
+test-static: test-lint test-types test-format  ## Run only static tests
+
 test-format: ## Run code formatting tests
 	black --check --diff $(BLACK_OPTS)
 
 test-lint: ## Run code linting tests
-	pylint --errors-only --enable=unused-import,unused-argument --ignore=templates ${SRC_DIRS}
+	pylint --errors-only --enable=unused-import,unused-argument --ignore=templates --ignore=docs/_ext ${SRC_DIRS}
 
 test-unit: ## Run unit tests
 	python -m unittest discover tests
 
 test-types: ## Check type definitions
-	mypy --exclude=templates --ignore-missing-imports --strict tutor/ tests/
+	mypy --exclude=templates --ignore-missing-imports --strict ${SRC_DIRS}
 
 test-pythonpackage: build-pythonpackage ## Test that package can be uploaded to pypi
 	twine check dist/tutor-$(shell make version).tar.gz
@@ -49,6 +51,9 @@ test-k8s: ## Validate the k8s format with kubectl. Not part of the standard test
 format: ## Format code automatically
 	black $(BLACK_OPTS)
 
+isort: ##  Sort imports. This target is not mandatory because the output may be incompatible with black formatting. Provided for convenience purposes.
+	isort --skip=templates ${SRC_DIRS}
+
 bootstrap-dev: ## Install dev requirements
 	pip install .
 	pip install -r requirements/dev.txt

bin/main.py

@@ -1,25 +1,17 @@
 #!/usr/bin/env python3
-from tutor.plugins import OfficialPlugin
+from tutor import hooks
 from tutor.commands.cli import main
+from tutor.plugins.v0 import OfficialPlugin
+
+
+@hooks.Actions.CORE_READY.add()
+def _discover_official_plugins() -> None:
+    # Manually discover plugins: that's because entrypoint plugins are not properly
+    # detected within the binary bundle.
+    with hooks.Contexts.PLUGINS.enter():
+        OfficialPlugin.discover_all()
 
-# Manually install plugins (this is for creating the bundle)
-for plugin_name in [
-    "android",
-    "discovery",
-    "ecommerce",
-    "forum",
-    "license",
-    "mfe",
-    "minio",
-    "notes",
-    "richie",
-    "webui",
-    "xqueue",
-]:
-    try:
-        OfficialPlugin.load(plugin_name)
-    except ImportError:
-        pass
 
 if __name__ == "__main__":
+    # Call the regular main function, which will not detect any entrypoint plugin
     main()

docs/Makefile

@@ -11,4 +11,7 @@ browse:
 	sensible-browser _build/html/index.html
 
 watch: build browse
-	while true; do inotifywait -e modify *.rst */*.rst */*/*.rst ../*.rst conf.py; $(MAKE) build || true; done
+	while true; do $(MAKE) wait-for-change build || true; done
+
+wait-for-change:
+	inotifywait -e modify $(shell find . -name "*.rst") ../*.rst ../tutor/hooks/*.py conf.py

docs/_ext/tutordocs.py

@@ -0,0 +1,14 @@
+"""
+This module is heavily inspired by Django's djangodocs.py:
+https://github.com/django/django/blob/main/docs/_ext/djangodocs.py
+"""
+from sphinx.application import Sphinx
+
+
+def setup(app: Sphinx) -> None:
+    # https://www.sphinx-doc.org/en/master/extdev/appapi.html#sphinx.application.Sphinx.add_crossref_type
+    app.add_crossref_type(
+        directivename="patch",
+        rolename="patch",
+        indextemplate="pair: %s; patch",
+    )

docs/conf.py

@@ -27,6 +27,10 @@
 exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
 pygments_style = None
 
+# Autodocumentation of modules
+extensions.append("sphinx.ext.autodoc")
+autodoc_typehints = "description"
+
 # -- Sphinx-Click configuration
 # https://sphinx-click.readthedocs.io/
 extensions.append("sphinx_click")
@@ -108,5 +112,10 @@ def youtube(
     ]
 
 
-youtube.content = True
-docutils.parsers.rst.directives.register_directive("youtube", youtube)
+# Tutor's own extension
+sys.path.append(os.path.join(os.path.dirname(__file__), "_ext"))
+extensions.append("tutordocs")
+
+
+setattr(youtube, "content", True)
+docutils.parsers.rst.directives.register_directive("youtube", youtube)  # type: ignore

docs/index.rst

@@ -22,9 +22,9 @@
    gettingstarted
    run
    configuration
-   plugins
-   reference
-   tutorials
+   plugins/index
+   reference/index
+   tutorials/index
    troubleshooting
    tutor
    faq
@@ -59,6 +59,6 @@ This work is licensed under the terms of the `GNU Affero General Public License
 
 The AGPL license covers the Tutor code, including the Dockerfiles, but not the content of the Docker images which can be downloaded from https://hub.docker.com. Software other than Tutor provided with the docker images retain their original license.
 
-The Tutor plugin system is licensed under the terms of the `Apache License, Version 2.0 <https://opensource.org/licenses/Apache-2.0>`__.
+The Tutor plugin and hooks system is licensed under the terms of the `Apache License, Version 2.0 <https://opensource.org/licenses/Apache-2.0>`__.
 
 © 2021 Tutor is a registered trademark of SASU NULI NULI. All Rights Reserved.

docs/intro.rst

@@ -70,6 +70,8 @@ Urls:
 
 The platform is reset every day at 9:00 AM, `Paris (France) time <https://time.is/Paris>`__, so feel free to try and break things as much as you want.
 
+.. _how_does_tutor_work:
+
 How does Tutor work?
 --------------------
 

docs/plugins/examples.rst

@@ -1,62 +1,87 @@
 .. _plugins_examples:
 
-Examples of Tutor plugins
-=========================
+========
+Examples
+========
 
 The following are simple examples of :ref:`Tutor plugins <plugins>` that can be used to modify the behaviour of Open edX.
 
 Skip email validation for new users
------------------------------------
+===================================
 
 ::
 
-    name: skipemailvalidation
-    version: 0.1.0
-    patches:
-      common-env-features: |
-        "SKIP_EMAIL_VALIDATION": true
+    from tutor import hooks
+
+    hooks.Filters.ENV_PATCHES.add_item(
+        (
+            "common-env-features",
+            """
+    "SKIP_EMAIL_VALIDATION": true
+    """"
+        )
+    )
 
 Enable bulk enrollment view in the LMS
---------------------------------------
+======================================
 
 ::
 
-    name: enablebulkenrollmentview
-    version: 0.1.0
-    patches:
-      lms-env-features: |
-        "ENABLE_BULK_ENROLLMENT_VIEW": true
+    from tutor import hooks
+
+    hooks.Filters.ENV_PATCHES.add_item(
+        (
+            "lms-env-features",
+            """
+    "ENABLE_BULK_ENROLLMENT_VIEW": true
+    """
+        )
+    )
 
 Enable Google Analytics
------------------------
+=======================
 
 ::
 
-    name: googleanalytics
-    version: 0.1.0
-    patches:
-      openedx-common-settings: |
-        # googleanalytics special settings
-        GOOGLE_ANALYTICS_ACCOUNT = "UA-your-account"
-        GOOGLE_ANALYTICS_TRACKING_ID = "UA-your-tracking-id"
+    from tutor import hooks
+
+    hooks.Filters.ENV_PATCHES.add_item(
+        (
+            "openedx-common-settings",
+            """
+  # googleanalytics special settings
+  GOOGLE_ANALYTICS_ACCOUNT = "UA-your-account"
+  GOOGLE_ANALYTICS_TRACKING_ID = "UA-your-tracking-id"
+  """
+        )
+    )
 
 Enable SAML authentication
---------------------------
+==========================
 
 ::
 
-    name: saml
-    version: 0.1.0
-    patches:
-      common-env-features: |
-        "ENABLE_THIRD_PARTY_AUTH": true
-
-      openedx-lms-common-settings: |
-        # saml special settings
-        AUTHENTICATION_BACKENDS += ["common.djangoapps.third_party_auth.saml.SAMLAuthBackend", "django.contrib.auth.backends.ModelBackend"]
-
-      openedx-auth: |
-        "SOCIAL_AUTH_SAML_SP_PRIVATE_KEY": "yoursecretkey",
-        "SOCIAL_AUTH_SAML_SP_PUBLIC_CERT": "yourpubliccert"
+    from tutor import hooks
+
+    hooks.Filters.ENV_PATCHES.add_items([
+        (
+            "common-env-features",
+            '"ENABLE_THIRD_PARTY_AUTH": true',
+        ),
+        (
+            "openedx-lms-common-settings:",
+            """
+    # saml special settings
+    AUTHENTICATION_BACKENDS += ["common.djangoapps.third_party_auth.saml.SAMLAuthBackend", "django.contrib.auth.backends.ModelBackend"]
+    """
+        ),
+        (
+            "openedx-auth",
+            """
+    "SOCIAL_AUTH_SAML_SP_PRIVATE_KEY": "yoursecretkey",
+    "SOCIAL_AUTH_SAML_SP_PUBLIC_CERT": "yourpubliccert"
+            """
+        ),
+    ])
 
 Do not forget to replace "yoursecretkey" and "yourpubliccert" with your own values.

docs/plugins/index.rst

@@ -0,0 +1,11 @@
+
+=======
+Plugins
+=======
+
+.. toctree::
+   :maxdepth: 2
+
+   intro
+   examples
+   v0/index

docs/plugins.rst → docs/plugins/intro.rst

@@ -1,7 +1,8 @@
 .. _plugins:
 
-Plugins
-=======
+============
+Introduction
+============
 
 Tutor comes with a plugin system that allows anyone to customise the deployment of an Open edX platform very easily. The vision behind this plugin system is that users should not have to fork the Tutor repository to customise their deployments. For instance, if you have created a new application that integrates with Open edX, you should not have to describe how to manually patch the platform settings, ``urls.py`` or ``*.env.json`` files. Instead, you can create a "tutor-myapp" plugin for Tutor. Then, users will start using your application in three simple steps::
 
@@ -12,12 +13,10 @@ Tutor comes with a plugin system that allows anyone to customise the deployment
     # 3) Reconfigure and restart the platform
     tutor local quickstart
 
-For simple changes, it may be extremely easy to create a Tutor plugin: even non-technical users may get started with :ref:`simple YAML plugins <plugins_yaml>`.
+For simple changes, it may be extremely easy to create a Tutor plugin: even non-technical users may get started with our :ref:`plugin_development_tutorial` tutorial. We also provide a list of :ref:`simple example plugins <plugins_examples>`.
 
-In the following, we learn how to use and create Tutor plugins.
-
-Commands
---------
+Plugin commands cheatsheet
+==========================
 
 List installed plugins::
 
@@ -32,19 +31,11 @@ After enabling or disabling a plugin, the environment should be re-generated wit
 
     tutor config save
 
+The full plugins CLI is described in the :ref:`reference documentation <cli_plugins>`.
+
 .. _existing_plugins:
 
 Existing plugins
-----------------
+================
 
 Officially-supported plugins are listed on the `Overhang.IO <https://overhang.io/tutor/plugins>`__ website.
-
-Plugin development
-------------------
-
-.. toctree::
-   :maxdepth: 2
-
-   plugins/api
-   plugins/gettingstarted
-   plugins/examples