From efd8da11c944e200413b613b3e24afa5323f7466 Mon Sep 17 00:00:00 2001 From: Bryann Valderrama <64033729+BryanttV@users.noreply.github.com> Date: Tue, 18 Feb 2025 08:55:03 -0500 Subject: [PATCH] docs: add linkcode extension (#261) --- CHANGELOG.rst | 9 +++ docs/common_refs.rst | 8 ++ docs/concepts/openedx-filters.rst | 13 ++-- docs/conf.py | 76 +++++++++++++++++++ docs/how-tos/create-a-new-filter.rst | 9 +-- docs/how-tos/create-a-pipeline-step.rst | 20 +++-- docs/reference/django-plugins-and-filters.rst | 4 +- openedx_filters/__init__.py | 2 +- 8 files changed, 117 insertions(+), 24 deletions(-) create mode 100644 docs/common_refs.rst diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 18b48a7..a9d7aa3 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -14,6 +14,15 @@ Change Log Unreleased ---------- +[2.0.1] - 2025-02-18 +-------------------- + +Changed +~~~~~~~ + +* Added linkcode Sphinx extension to the documentation. +* Added common_refs.rst file to reuse common references in the documentation. + [2.0.0] - 2025-02-17 --------------------- diff --git a/docs/common_refs.rst b/docs/common_refs.rst new file mode 100644 index 0000000..b2e1fd7 --- /dev/null +++ b/docs/common_refs.rst @@ -0,0 +1,8 @@ +.. _Tutor: https://docs.tutor.edly.io/ + +.. Replaces +.. |PipelineStep| replace:: :class:`PipelineStep ` +.. |PipelineStep.run_filter| replace:: :class:`run_filter ` +.. |CourseEnrollmentStarted| replace:: :class:`CourseEnrollmentStarted ` +.. |OpenEdxPublicFilter| replace:: :class:`OpenEdxPublicFilter ` +.. |OpenEdxPublicFilter.run_pipeline| replace:: :class:`run_pipeline ` diff --git a/docs/concepts/openedx-filters.rst b/docs/concepts/openedx-filters.rst index 8a09c11..115f68b 100644 --- a/docs/concepts/openedx-filters.rst +++ b/docs/concepts/openedx-filters.rst @@ -1,3 +1,5 @@ +.. include:: ../common_refs.rst + Open edX Filters ================ @@ -18,7 +20,7 @@ How do Open edX Filters work? Open edX Filters are implemented using an accumulative pipeline mechanism, which executes a series of functions in a specific order. Each function in the pipeline receives the output of the previous function as input, allowing developers to build complex processing logic by chaining multiple functions together. The pipeline ensures that the order of execution is maintained and that the result of a previous function is available to the current one in the form of a pipeline. -This pipeline mechanism is implemented by the `OpenEdxPublicFilter`_ class, which provides the necessary tools to fulfill the Open edX Filters requirements mentioned previously, such as ordered execution, configurability, interchangeable functions, argument definition, and cumulative behavior. This enables filters to modify the flow of the application dynamically during runtime based on predefined business logic or conditions. You can review the :doc:`Open edX Filters Tooling <../reference/filters-tooling>` for more information on the available methods and classes. +This pipeline mechanism is implemented by the |OpenEdxPublicFilter| class, which provides the necessary tools to fulfill the Open edX Filters requirements mentioned previously, such as ordered execution, configurability, interchangeable functions, argument definition, and cumulative behavior. This enables filters to modify the flow of the application dynamically during runtime based on predefined business logic or conditions. You can review the :doc:`Open edX Filters Tooling <../reference/filters-tooling>` for more information on the available methods and classes. Architectural Diagram ********************* @@ -33,7 +35,7 @@ Components ~~~~~~~~~~ #. Application (caller): The component that calls the filter during its execution, triggering the pipeline to process the input data. Developers may have added this call to a part of the application to include different behaviors. E.g., a user enrolls in a course, triggering the `CourseEnrollmentStarted filter`_. -#. OpenEdxPublicFilter: The class that implements all methods used to manage the execution of the filter. +#. |OpenEdxPublicFilter|: The class that implements all methods used to manage the execution of the filter. #. PipelineStep1...N: The pipeline steps that are executed in sequence, each processing the input data and returning potentially modified data. These steps are defined by the developer to introduce additional behaviors. E.g., a pipeline step that checks user eligibility for enrollment. Workflow @@ -43,7 +45,7 @@ Workflow #. The caller passes the input data to the filter through the ``run_filter`` method, this data are in-memory platform objects that the filter will process. -#. The ``run_filter`` method of the filter calls the ``OpenEdxPublicFilter.run_pipeline`` method under the hood, which manages the execution of the filter's pipeline. +#. The ``run_filter`` method of the filter calls the |OpenEdxPublicFilter.run_pipeline| method under the hood, which manages the execution of the filter's pipeline. #. This method retrieves the configuration from ``OPEN_EDX_FILTERS_CONFIG``, which defines a list of N functions :math:`f_1, f_2, \ldots, f_{n}` that will be executed. @@ -53,7 +55,7 @@ Workflow #. Each subsequent function receives the output from the previous function and returns its modified output until all functions have been executed. -#. At any point in the pipeline, a developer can halt execution by raising an exception, based on conditions defined in the processing logic, to stop the application flow. Let's assume that :math:`f_{2}` raises an exception instead of returning the modified arguments ``kwargs_2``. In this case, the pipeline stops, and the ``OpenEdxPublicFilter.run_pipeline`` method raises the exception to the caller as the final output. From there the caller can handle the exception as needed. +#. At any point in the pipeline, a developer can halt execution by raising an exception, based on conditions defined in the processing logic, to stop the application flow. Let's assume that :math:`f_{2}` raises an exception instead of returning the modified arguments ``kwargs_2``. In this case, the pipeline stops, and the |OpenEdxPublicFilter.run_pipeline| method raises the exception to the caller as the final output. From there the caller can handle the exception as needed. #. If no exceptions are raised, the pipeline continues executing the functions until the final function :math:`f_{n}` has been executed. @@ -68,7 +70,7 @@ Here's an example of the `CourseEnrollmentStarted filter`_ in action: #. A user enrolls in a course, triggering the `CourseEnrollmentStarted filter`_ by calling the ``run_filter`` method with the enrollment details. This filter processes information about the user, course, and enrollment details. -#. The ``run_pipeline`` method executes a series of functions configured in ``OPEN_EDX_FILTERS_CONFIG``, e.g. checking user eligibility for enrollment or updating the enrollment status in a third-party system. +#. The |OpenEdxPublicFilter.run_pipeline| method executes a series of functions configured in ``OPEN_EDX_FILTERS_CONFIG``, e.g. checking user eligibility for enrollment or updating the enrollment status in a third-party system. #. Each function can modify the input data or halt the process based on business logic, e.g. denying enrollment if the user is ineligible. @@ -89,5 +91,4 @@ For more information on how to use Open edX Filters, refer to the :doc:`how-tos .. _Django Signals Documentation: https://docs.djangoproject.com/en/4.2/topics/signals/ .. _CourseEnrollmentStarted filter: https://github.com/openedx/edx-platform/blob/master/common/djangoapps/student/models/course_enrollment.py#L719-L724 .. _Python Social Auth: https://python-social-auth.readthedocs.io/en/latest/pipeline.html -.. _OpenEdxPublicFilter: https://github.com/openedx/openedx-filters/blob/main/openedx_filters/tooling.py#L14-L15 .. _Open edX Django plugin: https://edx.readthedocs.io/projects/edx-django-utils/en/latest/plugins/readme.html diff --git a/docs/conf.py b/docs/conf.py index f23c413..ef71801 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -10,9 +10,12 @@ # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. # +import inspect import os import re import sys +from os.path import dirname, relpath + sys.path.insert(0, os.path.abspath('..')) # -- Project information ----------------------------------------------------- @@ -39,6 +42,7 @@ 'sphinx.ext.autosummary', 'sphinx.ext.intersphinx', 'sphinx.ext.napoleon', + 'sphinx.ext.linkcode', ] # Add any paths that contain templates here, relative to this directory. @@ -135,3 +139,75 @@ None, ), } + +# Linkcode Extension Configuration + +REPO_URL = "https://github.com/openedx/openedx-filters/blob/main" + +def linkcode_resolve(domain: str, info: dict[str, str]) -> str | None: + """ + Resolves source code links for Python objects in Sphinx documentation. + This function is based on the `linkcode_resolve` function in the SciPy project. + + Args: + domain (str): The language domain of the object. Only processes Python objects ('py') + info (dict[str, str]): Dictionary containing information about the object to link. + Must contain: + - 'module': Name of the module containing the object + - 'fullname': Complete name of the object including its path + + Returns: + str | None: URL to the source code on GitHub with specific line numbers, + or None if the link cannot be resolved + """ + if domain != "py": + return None + + modname = info["module"] + fullname = info["fullname"] + + submod = sys.modules.get(modname) + if submod is None: + return None + + obj = submod + for part in fullname.split("."): + try: + obj = getattr(obj, part) + except Exception: + return None + + # Use the original function object if it is wrapped. + while hasattr(obj, "__wrapped__"): + obj = obj.__wrapped__ + + # Get the file path where the object is defined + try: + # Try to get the file path of the object directly + file_path = inspect.getsourcefile(obj) + except Exception: + try: + # If that fails, try to get the file path of the module where the object is defined + file_path = inspect.getsourcefile(sys.modules[obj.__module__]) + except Exception: + # If both attempts fail, set file_path to None + file_path = None + if not file_path: + return None + + try: + source, start_line = inspect.getsourcelines(obj) + except Exception: + start_line = None + + if start_line: + linespec = f"#L{start_line}-L{start_line + len(source) - 1}" + else: + linespec = "" + + import openedx_filters + + start_dir = os.path.abspath(os.path.join(dirname(openedx_filters.__file__), "..")) + file_path = relpath(file_path, start=start_dir).replace(os.path.sep, "/") + + return f"{REPO_URL}/{file_path}{linespec}" diff --git a/docs/how-tos/create-a-new-filter.rst b/docs/how-tos/create-a-new-filter.rst index c1fcab3..1b373a3 100644 --- a/docs/how-tos/create-a-new-filter.rst +++ b/docs/how-tos/create-a-new-filter.rst @@ -1,3 +1,5 @@ +.. include:: ../common_refs.rst + Create a New Open edX Filter with Long-Term Support #################################################### @@ -97,7 +99,7 @@ In our example, the filter arguments could include the user, course key, and enr Step 5: Implement the Filter Definition ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Implement the :term:`filter definition` by creating a new class that inherits from the `OpenEdxPublicFilter`_ class. The filter definition should implement the ``run_filter`` method, which defines the input and output behavior of the filter. The ``run_filter`` method should call the method `run_pipeline`_, passing down the input arguments and returning the final output of the filter. This class should be placed in the appropriate subdomain module in the library, in the ``filters.py`` file. +Implement the :term:`filter definition` by creating a new class that inherits from the |OpenEdxPublicFilter| class. The filter definition should implement the ``run_filter`` method, which defines the input and output behavior of the filter. The ``run_filter`` method should call the method |OpenEdxPublicFilter.run_pipeline|, passing down the input arguments and returning the final output of the filter. This class should be placed in the appropriate subdomain module in the library, in the ``filters.py`` file. .. note:: The input arguments of the ``run_filter`` method should match the arguments that the triggering logic provides. The output of the filter should be consistent with the behavior that the filter intends to modify. Usually, the output is the modified data or the original data if no modifications are needed. @@ -138,7 +140,7 @@ In our example, the filter definition could be implemented as follows: - The ``filter_type`` attribute should be set to the filter type that was identified in the previous steps. This attribute is used to identify the filter in the :term:`filter configuration`. - The ``PreventEnrollment`` class is a custom exception that is raised when the filter should halt the application behavior. -- The ``run_filter`` method is the main method of the filter that is called when the filter is triggered. The method should call the ``run_pipeline`` method, passing down the input arguments and returning the final output of the filter. +- The ``run_filter`` method is the main method of the filter that is called when the filter is triggered. The method should call the |OpenEdxPublicFilter.run_pipeline| method, passing down the input arguments and returning the final output of the filter. - Use arguments names that are consistent with the triggering logic to avoid confusion and improve readability. .. note:: Implement exceptions that are related to the filter behavior and specify how the filter should modify the application behavior with each exception. The caller should handle each exception differently based the exceptions purpose. For example, the caller should halt the application behavior when the ``PreventEnrollment`` exception is raised. @@ -179,8 +181,5 @@ After implementing the filter, you should continue the contribution process by c For more details on how the contribution flow works, refer to the :doc:`docs.openedx.org:developers/concepts/hooks_extension_framework` documentation. -.. _Tutor: https://docs.tutor.edly.io/ .. _Add Extensibility Mechanism to IDV to Enable Integration of New IDV Vendor Persona: https://openedx.atlassian.net/wiki/spaces/OEPM/pages/4307386369/Proposal+Add+Extensibility+Mechanisms+to+IDV+to+Enable+Integration+of+New+IDV+Vendor+Persona -.. _OpenEdxPublicFilter: https://github.com/openedx/openedx-filters/blob/main/openedx_filters/tooling.py#L14 -.. _run_pipeline: https://github.com/openedx/openedx-filters/blob/main/openedx_filters/tooling.py#L164 .. _test_filters.py: https://github.com/openedx/edx-platform/blob/master/common/djangoapps/student/tests/test_filters.py#L114-L190 diff --git a/docs/how-tos/create-a-pipeline-step.rst b/docs/how-tos/create-a-pipeline-step.rst index 529cbae..4a073ff 100644 --- a/docs/how-tos/create-a-pipeline-step.rst +++ b/docs/how-tos/create-a-pipeline-step.rst @@ -1,3 +1,5 @@ +.. include:: ../common_refs.rst + Create a Pipeline Step ###################### @@ -25,7 +27,7 @@ Step 1: Understand your Use Case and Identify the Filter to Use Before creating a pipeline step, you should understand your use case for the filter and the specific logic you want to implement in the pipeline step. In our example, we want to prevent users from enrolling in a course if they do not have a valid email address. We will create a pipeline step that checks if the user's email address is valid and raise an exception if it is not. -You should review the :doc:`list of filters <../reference/filters>` available in the Open edX platform and identify the filter that best fits your use case. In our example, we will use the `CourseEnrollmentStarted filter`_ to implement the logic for our use case. You should review the filter's arguments to understand the data that will be passed to the pipeline step and the expected output. This will help you define the pipeline step's logic and signature. +You should review the :doc:`list of filters <../reference/filters>` available in the Open edX platform and identify the filter that best fits your use case. In our example, we will use the |CourseEnrollmentStarted| filter to implement the logic for our use case. You should review the filter's arguments to understand the data that will be passed to the pipeline step and the expected output. This will help you define the pipeline step's logic and signature. Step 2: Install Open edX Filters ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -41,11 +43,11 @@ This will mainly make the filters available for your CI/CD pipeline and local de Step 3: Create a Pipeline Step ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -A :term:`pipeline step` is a class that inherits from the base class `PipelineStep`_ and defines specific logic within its `run_filter`_ method. The ``run_filter`` method is executed by the pipeline tooling when the filter is triggered. To create a pipeline step, you should: +A :term:`pipeline step` is a class that inherits from the base class |PipelineStep| and defines specific logic within its |PipelineStep.run_filter| method. The ``run_filter`` method is executed by the pipeline tooling when the filter is triggered. To create a pipeline step, you should: 1. Create a new Python module for the pipeline step called ``pipeline.py``. Pipeline steps are usually implemented in a `Open edX Django plugins`_, so you should create the module in the plugin's directory. -2. Create a new class for the pipeline step that inherits from the base class `PipelineStep`_. -3. Implement the logic for the pipeline step within the `run_filter`_ method. The method signature should match the filter's signature to ensure compatibility with the pipeline tooling. In our example, the method should accept the user, course key, and enrollment mode as arguments and return the same arguments if the email address is valid. If the email address is not valid, the method should raise an exception. +2. Create a new class for the pipeline step that inherits from the base class |PipelineStep|. +3. Implement the logic for the pipeline step within the |PipelineStep.run_filter| method. The method signature should match the filter's signature to ensure compatibility with the pipeline tooling. In our example, the method should accept the user, course key, and enrollment mode as arguments and return the same arguments if the email address is valid. If the email address is not valid, the method should raise an exception. 4. You can take an iterative approach to developing the pipeline step by testing it locally and making changes as needed. In our example, the pipeline step could look like this: @@ -67,8 +69,8 @@ In our example, the pipeline step could look like this: "mode": mode, } -- In this example, we create a new class called ``CheckValidEmailPipelineStep`` that inherits from the base class `PipelineStep`_. -- We implement the logic for the pipeline step within the `run_filter`_ method. The method checks if the user's email address is valid using the ``is_user_email_allowed`` method and raises an exception if it is not. If the email address is valid, the method returns the user, course key, and enrollment mode in a dictionary. +- In this example, we create a new class called ``CheckValidEmailPipelineStep`` that inherits from the base class |PipelineStep|. +- We implement the logic for the pipeline step within the |PipelineStep.run_filter| method. The method checks if the user's email address is valid using the ``is_user_email_allowed`` method and raises an exception if it is not. If the email address is valid, the method returns the user, course key, and enrollment mode in a dictionary. - The method signature matches the filter's signature, accepting the user, course key, and enrollment mode as arguments and returning the same arguments if the email address is valid. You can also return an empty dictionary if you don't need to modify the data. Consider the following when creating a pipeline step: @@ -83,7 +85,7 @@ Step 4: Configure the Pipeline for the Filter After creating the pipeline step, you need to configure the pipeline for the filter in the :term:`filter configuration`. The configuration settings are specific for each :term:`filter type` and define the pipeline steps to be executed when the filter is triggered. You should add the path to the pipeline step class in the filter's pipeline configuration. -In our example, we will configure the pipeline for the `CourseEnrollmentStarted filter`_ to include the pipeline step we created. The configuration should look like this: +In our example, we will configure the pipeline for the |CourseEnrollmentStarted| filter to include the pipeline step we created. The configuration should look like this: .. code-block:: python @@ -139,8 +141,4 @@ After testing the pipeline step, you should debug and iterate on the implementat .. note:: The default behavior of the pipeline tooling is to fail silently if a runtime exception is raised in a pipeline step. You can configure the filter to raise an exception when the pipeline step fails by setting ``fail_silently: False`` in the filter configuration. This will help you identify issues early and take appropriate action to resolve them. :term:`Filter Exceptions` will always be raised in the pipeline and will halt the execution of the pipeline. You can use exceptions to control the flow of the pipeline and handle specific scenarios in the pipeline step. In our example, we raise an exception when the user's email address is not valid to prevent them from enrolling in the course. The exceptions considered by the ``fail_silently`` flag are runtime exceptions that are not intentionally raised by the developer during the filter's execution, use the configuration as you see fit. -.. _Tutor: https://docs.tutor.edly.io/ -.. _CourseEnrollmentStarted filter: https://github.com/openedx/openedx-filters/blob/main/openedx_filters/learning/filters.py#L145-L170 -.. _PipelineStep: https://github.com/openedx/openedx-filters/blob/main/openedx_filters/filters.py#L10-L77 .. _Open edX Django plugins: https://docs.openedx.org/en/latest/developers/concepts/platform_overview.html#new-plugin -.. _run_filter: https://github.com/openedx/openedx-filters/blob/main/openedx_filters/filters.py#L60-L77 diff --git a/docs/reference/django-plugins-and-filters.rst b/docs/reference/django-plugins-and-filters.rst index 27390b6..10f085a 100644 --- a/docs/reference/django-plugins-and-filters.rst +++ b/docs/reference/django-plugins-and-filters.rst @@ -1,3 +1,5 @@ +.. include:: ../common_refs.rst + Django Plugins and Filters ########################## @@ -37,5 +39,5 @@ file. The dictionary has the following structure: Create pipeline steps ********************* -In your own plugin, you can create your own :term:`pipeline steps` by inheriting from ``PipelineStep`` and implementing the +In your own plugin, you can create your own :term:`pipeline steps` by inheriting from |PipelineStep| and implementing the ``run_filter`` method. You can find examples of :term:`pipeline steps` in the ``openedx-filters-samples`` repository. See :doc:`/quickstarts/index` for more details. diff --git a/openedx_filters/__init__.py b/openedx_filters/__init__.py index 4d9c705..359927f 100644 --- a/openedx_filters/__init__.py +++ b/openedx_filters/__init__.py @@ -3,4 +3,4 @@ """ from openedx_filters.filters import * -__version__ = "2.0.0" +__version__ = "2.0.1"