From 89c85cc8052bc03a0b100a54a248b403418a2618 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Wed, 18 Dec 2024 12:20:38 +0100 Subject: [PATCH 01/38] docs: refactor names for consistency in how-tos --- docs/concepts/event-bus.rst | 2 +- docs/concepts/openedx-events.rst | 2 +- ...-new-event-bus-concrete-implementation.rst | 4 +- docs/how-tos/adding-events-to-a-service.rst | 39 ----- docs/how-tos/creating-new-events.rst | 134 ------------------ docs/how-tos/index.rst | 6 +- docs/how-tos/using-events.rst | 96 ------------- docs/reference/architecture-subdomains.rst | 18 ++- docs/reference/real-life-use-cases.rst | 2 +- 9 files changed, 24 insertions(+), 279 deletions(-) delete mode 100644 docs/how-tos/adding-events-to-a-service.rst delete mode 100644 docs/how-tos/creating-new-events.rst delete mode 100644 docs/how-tos/using-events.rst diff --git a/docs/concepts/event-bus.rst b/docs/concepts/event-bus.rst index 9c84cceb..93b22d81 100644 --- a/docs/concepts/event-bus.rst +++ b/docs/concepts/event-bus.rst @@ -119,7 +119,7 @@ How is the Open edX Event Bus Used? The event bus is used to broadcast Open edX Events to multiple services, allowing them to react to changes or actions in the system. -We encourage you to review the :doc:`../reference/real-life-use-cases` page for examples of how the event bus is used in the Open edX ecosystem by the community. Also, see the :doc:`../how-tos/using-the-event-bus` guide to get start sending events to the event bus. +We encourage you to review the :doc:`../reference/real-life-use-cases` page for examples of how the event bus is used in the Open edX ecosystem by the community. Also, see the :doc:`../how-tos/use-the-event-bus-to-broadcast-and-consume-events` guide to get start sending events to the event bus. .. _general_signal_handler: https://github.com/openedx/openedx-events/blob/main/openedx_events/apps.py#L16-L44 .. _EventProducer: https://github.com/openedx/openedx-events/blob/main/openedx_events/event_bus/__init__.py#L71-L91 diff --git a/docs/concepts/openedx-events.rst b/docs/concepts/openedx-events.rst index 350dcea4..2d2a5f0f 100644 --- a/docs/concepts/openedx-events.rst +++ b/docs/concepts/openedx-events.rst @@ -75,7 +75,7 @@ How are Open edX Events used? As mentioned previously, developers can listen to Open edX Events by registering signal receivers from their Open edX Django plugins that respond to the emitted events or by using the :doc:`../concepts/event-bus` to send events to external services. -For more information on using Open edX Events, refer to the :doc:`../how-tos/using-events` how-to guide. We also encourage you to explore the :doc:`../reference/real-life-use-cases` section for real-life examples of how Open edX Events are used by the community. +For more information on using Open edX Events, refer to the :doc:`../how-tos/create-a-new-event` how-to guide. We also encourage you to explore the :doc:`../reference/real-life-use-cases` section for real-life examples of how Open edX Events are used by the community. .. _Django Signals Documentation: https://docs.djangoproject.com/en/4.2/topics/signals/ .. _triggering the COURSE_ENROLLMENT_CREATED event: https://github.com/openedx/edx-platform/blob/master/common/djangoapps/student/models/course_enrollment.py#L777-L795 diff --git a/docs/how-tos/add-new-event-bus-concrete-implementation.rst b/docs/how-tos/add-new-event-bus-concrete-implementation.rst index a4d21eac..b35deb1f 100644 --- a/docs/how-tos/add-new-event-bus-concrete-implementation.rst +++ b/docs/how-tos/add-new-event-bus-concrete-implementation.rst @@ -1,5 +1,5 @@ -How to add a new concrete implementation of the event bus -========================================================= +Add a New Concrete Implementation of the Event Bus +================================================== Context ------- diff --git a/docs/how-tos/adding-events-to-a-service.rst b/docs/how-tos/adding-events-to-a-service.rst deleted file mode 100644 index 532284c7..00000000 --- a/docs/how-tos/adding-events-to-a-service.rst +++ /dev/null @@ -1,39 +0,0 @@ -How to add an Open edX Event to a service -========================================= - -The next step after creating your first event in the Open edX Events library is to trigger the event in the service -you implemented it for. Here is a checklist of what we've done so far when including a new event to a service: - -- Add the openedx-events library to the service project. -- Import the events' data and definition into the place where the event be triggered. Remember the event's purpose when - choosing a place to send the new event. -- Add inline documentation with the ``event_implemented_name``. This matches the ``event_name`` in line documentation from the library. -- Refer to the service project's contribution guidelines and follow the instructions. Then, open a new pull request! - -Consider the addition of the event ``STUDENT_REGISTRATION_COMPLETED`` to edx-platform as an example: - -.. code-block:: python - - # Location openedx/core/djangoapps/user_authn/views/register.py - # .. event_implemented_name: COURSE_ENROLLMENT_CREATED - COURSE_ENROLLMENT_CREATED.send_event( - enrollment=CourseEnrollmentData( - user=UserData( - pii=UserPersonalData( - username=user.username, - email=user.email, - name=user.profile.name, - ), - id=user.id, - is_active=user.is_active, - ), - course=course_data, - mode=enrollment.mode, - is_active=enrollment.is_active, - creation_date=enrollment.created, - ) - ) - -If you want to know more about how the integration of the first events' batch went, check out the `PR 28266`_. - -.. _PR 28266: https://github.com/openedx/edx-platform/pull/28266 diff --git a/docs/how-tos/creating-new-events.rst b/docs/how-tos/creating-new-events.rst deleted file mode 100644 index a22971c0..00000000 --- a/docs/how-tos/creating-new-events.rst +++ /dev/null @@ -1,134 +0,0 @@ -How to create a new Open edX Event -================================== - -The mechanisms implemented by the Open edX Events library are supported and maintained by the Open edX community. -Therefore, we've put together a guide on how to add a new event to the library so future contributions are effective. - - -1. Propose the new event to the community ------------------------------------------ - -When creating a new event, you must justify its implementation. For example, you could create a post in Discuss, -send a message through slack or open a new issue in the library repository listing your use cases for it. Or even, -if you have time, you could accompany your proposal with the implementation of the event to illustrate its behavior. - -.. note:: - There is an open discussion about whether organization scoped events would also exist in openedx-events, - in which case this step would be optional. See issue `Organization scoped events `_. - -2. Place your event in an architecture subdomain -------------------------------------------------- - -As specified in the Architectural Decisions Record (ADR) events naming and versioning, the :term:`event definition ` needs an Open edX Architecture -Subdomain for: - -- The name of the event: ``{Reverse DNS}.{Architecture Subdomain}.{Subject}.{Action}.{Major Version}`` -- The package name where the definition will live, eg. ``learning/`` or ``content_authoring/``. - -For those reasons, after studying your new event purpose, you must place it in one of the subdomains already in use, or introduce a new subdomain: - -+-------------------+----------------------------------------------------------------------------------------------------+ -| Subdomain name | Description | -+===================+====================================================================================================+ -| Content Authoring | Allows educators to create, modify, package, annotate (tag), and share learning content. | -+-------------------+----------------------------------------------------------------------------------------------------+ -| Learning | Allows learners to consume content and perform actions in a learning activity on the platform. | -+-------------------+----------------------------------------------------------------------------------------------------+ - -New subdomains may require some discussion, because there does not yet exist and agreed upon set on subdomains. So we encourage you to start the conversation -as soon as possible through any of the communication channels available. - -Refer to `edX DDD Bounded Contexts `_ confluence page for more documentation on domain-driven design in the Open edX project. - -3. Create the data attributes for the event (OEP-49) ----------------------------------------------------- - -Events send `data attributes `_ when triggered. Therefore, when designing your new :term:`event definition ` you must -decide if an existent data class works for your use case or you must create a new one. If the answer is the latter, then try to answer: - -- Which attributes of the object are the most relevant? -- Which type are they? -- Are any of them optional/required? - -And with that information, create the new class justifying each decision. The class created in this step must comply -with: - -- It should be created in the `data.py` file, as described in the OEP-49, in the corresponding architectural subdomain. Refer to Naming Conventions ADR for more - on events subdomains. -- It should follow the naming conventions used across the other events definitions. - -Consider the user data representation as an example: - -.. code-block:: python - - @attr.s(frozen=True) - class CourseData: - """ - Attributes defined for Open edX Course Overview object. - - Arguments: - course_key (str): identifier of the Course object. - display_name (str): display name associated with the course. - start (datetime): start date for the course. - end (datetime): end date for the course. - """ - - course_key = attr.ib(type=CourseKey) - display_name = attr.ib(type=str, factory=str) - start = attr.ib(type=datetime, default=None) - end = attr.ib(type=datetime, default=None) - - - @attr.s(frozen=True) - class CourseEnrollmentData: - """ - Attributes defined for Open edX Course Enrollment object. - - Arguments: - user (UserData): user associated with the Course Enrollment. - course (CourseData): course where the user is enrolled in. - mode (str): course mode associated with the course. - is_active (bool): whether the enrollment is active. - creation_date (datetime): creation date of the enrollment. - created_by (UserData): if available, who created the enrollment. - """ - - user = attr.ib(type=UserData) - course = attr.ib(type=CourseData) - mode = attr.ib(type=str) - is_active = attr.ib(type=bool) - creation_date = attr.ib(type=datetime) - created_by = attr.ib(type=UserData, default=None) - -4. Create the event definition ------------------------------- - -Open edX Events are instances of the class OpenEdxPublicSignal, this instance represents the :term:`event definition ` that -specifies: - -- The :term:`event type ` which should follow the conventions in the Naming Conventions ADR. -- The events' payload, here you must use the class you decided on before. - -The definition created in this step must comply with: - -- It should be created in the ``signals.py`` file in the corresponding subdomain. Refer to Naming Conventions ADR for more - on events subdomains. -- It should follow the naming conventions specified in :doc:`../decisions/0002-events-naming-and-versioning`. -- It must be documented using in-line documentation with at least: ``event_type``, ``event_name``, ``event_description`` and - ``event_data``. See :doc:`../reference/in-line-code-annotations-for-an-event` for more information. - -Consider the course enrollment created event as an example: - -.. code-block:: python - - # Location openedx_events/learning/signals.py - # .. event_type: org.openedx.learning.course.enrollment.created.v1 - # .. event_name: COURSE_ENROLLMENT_CREATED - # .. event_description: emitted when the user's enrollment process is completed. - # .. event_data: CourseEnrollmentData - COURSE_ENROLLMENT_CREATED = OpenEdxPublicSignal( - event_type="org.openedx.learning.course.enrollment.created.v1", - data={ - "enrollment": CourseEnrollmentData, - } - ) diff --git a/docs/how-tos/index.rst b/docs/how-tos/index.rst index edb107f7..325d0bd6 100644 --- a/docs/how-tos/index.rst +++ b/docs/how-tos/index.rst @@ -5,9 +5,9 @@ How-tos :maxdepth: 1 :caption: Contents: - creating-new-events - adding-events-to-a-service - using-events + create-a-new-event + consume-an-event adding-event-bus-support-to-an-event + use-the-event-bus-to-broadcast-and-consume-events using-the-event-bus add-new-event-bus-concrete-implementation diff --git a/docs/how-tos/using-events.rst b/docs/how-tos/using-events.rst deleted file mode 100644 index 3ebdcb6e..00000000 --- a/docs/how-tos/using-events.rst +++ /dev/null @@ -1,96 +0,0 @@ -Using Open edX Events -===================== - -How to use ----------- - -Using openedx-events in your code is very straight forward. We can consider the -two possible cases, sending or receiving an event. - -Receiving events -^^^^^^^^^^^^^^^^ - -This is one of the most common use cases for plugins. The Open edX platform will send -an event and you want to react to it in your plugin. - -For this you need to: - -1. Include openedx-events in your dependencies. -2. Connect your :term:`receiver ` functions to the signals being sent. - -Connecting signals can be done using regular django syntax: - -.. code-block:: python - - from openedx_events.learning.signals import STUDENT_REGISTRATION_COMPLETED - - @receiver(STUDENT_REGISTRATION_COMPLETED) - def your_receiver_function(**kwargs): - # your implementation here - -.. note:: - You may encounter an older pattern for configuring signal receivers that is no longer recommended. This - outdated Django plugin configuration used ``plugin_app`` and ``signals_config`` in apps.py. - -.. warning:: - For non-trivial work, we encourage using asynchronous tasks in your receiver functions in order - to avoid affecting the performance of the service. - -Sending events -^^^^^^^^^^^^^^ - -Sending events requires you to import both the :term:`event definition ` as well as the -attr data classes that encapsulate the event data. - -.. code-block:: python - - from openedx_events.learning.data import UserData, UserPersonalData - from openedx_events.learning.signals import STUDENT_REGISTRATION_COMPLETED - - STUDENT_REGISTRATION_COMPLETED.send_event( - user=UserData( - pii=UserPersonalData( - username=user.username, - email=user.email, - name=user.profile.name, - ), - id=user.id, - is_active=user.is_active, - ), - ) - -You can do this both from the Open edX platform code as well as from an openedx -plugin. - -Testing events -^^^^^^^^^^^^^^ - -Testing your code in CI, specially for plugins is now possible without having to -import the complete Open edX platform as a dependency. - -To test your functions you need to include the openedx-events library in your -testing dependencies and make the signal connection in your test case. - -.. code-block:: python - - from openedx_events.learning.signals import STUDENT_REGISTRATION_COMPLETED - - def test_your_receiver(self): - STUDENT_REGISTRATION_COMPLETED.connect(your_function) - STUDENT_REGISTRATION_COMPLETED.send_event( - user=UserData( - pii=UserPersonalData( - username='test_username', - email='test_email@example.com', - name='test_name', - ), - id=1, - is_active=True, - ), - ) - - # run your assertions - -Changes in the openedx-events library that are not compatible with your code -should break this kind of test in CI and let you know you need to upgrade your -code. diff --git a/docs/reference/architecture-subdomains.rst b/docs/reference/architecture-subdomains.rst index b69adbc6..c65eb991 100644 --- a/docs/reference/architecture-subdomains.rst +++ b/docs/reference/architecture-subdomains.rst @@ -1,8 +1,21 @@ Architecture Subdomains ======================= -Here we list useful information about Open edX architecture subdomains and -their use in the Hooks Extension framework: +Currently, these are the `architecture subdomains`_ used by the Open edX Events library: + ++-------------------+----------------------------------------------------------------------------------------------------+ +| Subdomain name | Description | ++===================+====================================================================================================+ +| Content Authoring | Allows educators to create, modify, package, annotate (tag), and share learning content. | ++-------------------+----------------------------------------------------------------------------------------------------+ +| Learning | Allows learners to consume content and perform actions in a learning activity on the platform. | ++-------------------+----------------------------------------------------------------------------------------------------+ +| Analytics | Provides insights into learner behavior and course performance. | ++-------------------+----------------------------------------------------------------------------------------------------+ +| Enterprise | Provides tools for organizations to manage their learners and courses. | ++-------------------+----------------------------------------------------------------------------------------------------+ + +Here we list useful information about Open edX architecture subdomains and their use in the Hooks Extension framework: - `Events Naming and Versioning`_ - `Notes on events design and subdomains`_ @@ -15,3 +28,4 @@ their use in the Hooks Extension framework: .. _`Subdomains from OEP-41`: https://docs.openedx.org/projects/openedx-proposals/en/latest/architectural-decisions/oep-0041-arch-async-server-event-messaging.html#subdomain-from-domain-driven-design .. _`Message Content Data Guidelines`: https://docs.openedx.org/projects/openedx-proposals/en/latest/architectural-decisions/oep-0041-arch-async-server-event-messaging.html?highlight=subdomain#message-content-data-guidelines .. _`Notes on events design and subdomains`: https://github.com/openedx/openedx-events/issues/72#issuecomment-1179291340 +.. _architecture subdomains: https://microservices.io/patterns/decomposition/decompose-by-subdomain.html diff --git a/docs/reference/real-life-use-cases.rst b/docs/reference/real-life-use-cases.rst index 10eeebb7..2b778381 100644 --- a/docs/reference/real-life-use-cases.rst +++ b/docs/reference/real-life-use-cases.rst @@ -16,7 +16,7 @@ The following list of real-life use cases showcases the different ways Open edX Cross-services communication **************************** -As mentioned in :doc:`../concepts/event-bus`, the suggested strategy for cross-service communication in the Open edX ecosystem is through an event-based architecture implemented via the :doc:`../concepts/event-bus`. This functionality used for asynchronous communication between services is built on top of sending Open edX Events (Open edX-specific Django signals) within a service. For more details on the Event Bus, please see :doc:`../how-tos/using-the-event-bus`. +As mentioned in :doc:`../concepts/event-bus`, the suggested strategy for cross-service communication in the Open edX ecosystem is through an event-based architecture implemented via the :doc:`../concepts/event-bus`. This functionality used for asynchronous communication between services is built on top of sending Open edX Events (Open edX-specific Django signals) within a service. For more details on the Event Bus, please see :doc:`../how-tos/use-the-event-bus-to-broadcast-and-consume-events`. Here are some examples of how the Event Bus can be used to facilitate communication between IDAs: From 773b77830e3dc4049afe4620434545d38d7a2811 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Thu, 19 Dec 2024 17:54:47 +0100 Subject: [PATCH 02/38] docs: add docs about creating and consuming events --- docs/how-tos/consume-an-event.rst | 72 ++++++ docs/how-tos/create-a-new-event.rst | 224 ++++++++++++++++++ ...nt-bus-to-broadcast-and-consume-events.rst | 87 +++++++ 3 files changed, 383 insertions(+) create mode 100644 docs/how-tos/consume-an-event.rst create mode 100644 docs/how-tos/create-a-new-event.rst create mode 100644 docs/how-tos/use-the-event-bus-to-broadcast-and-consume-events.rst diff --git a/docs/how-tos/consume-an-event.rst b/docs/how-tos/consume-an-event.rst new file mode 100644 index 00000000..47abd56d --- /dev/null +++ b/docs/how-tos/consume-an-event.rst @@ -0,0 +1,72 @@ +Consume an Open edX Event +========================= + +You have two ways of consuming an Open edX event, within the same service or in a different service. In this guide, we will show you how to consume an event within the same service. For consuming events across services, see :doc:`../how-tos/use-the-event-bus-to-broadcast-and-consume-events`. + +.. note:: We encourage you to also consider the practices outlined in the :doc:`../decisions/0016-event-design-practices` ADR for event consumption. + +Throughout this guide, we will use an example of creating an event handler that will execute when a user enrolls in a course from the course about page to better illustrate the steps involved in creating a consumer for an event. + +Setup +----- + +To consume an event within the same service, follow these steps: + +Step 1: Install Open edX Events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +First, add the ``openedx-events`` plugin into your dependencies so the library's environment recognizes the event you want to consume. You can install ``openedx-events`` by running: + +.. code-block:: bash + + pip install openedx-events + +This will mainly make the events available for your CI/CD pipeline and local development environment. If you are using the Open edX platform, the library should be already be installed in the environment so no need to install it. + +Step 2: Create a Event Receiver and Connect it to the Event +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +An :term:`Event Receiver` is simply a function that listens for a specific event and executes custom logic in response to the event being triggered. You can create an event receiver by using the Django signal receivers decorator. Here's an example of an event receiver that listens for the ``COURSE_ENROLLMENT_CREATED`` event and creates a notification preference for the user: + +.. code-block:: python + + from openedx_events import COURSE_ENROLLMENT_CREATED + from django.dispatch import receiver + + @receiver(COURSE_ENROLLMENT_CREATED) + def create_notification_preference(signal, sender, enrollment, metadata, **kwargs): + # Custom logic to create a notification preference for the user + pass + +Now, the django dispatcher will call the ``create_notification_preference`` function when the ``COURSE_ENROLLMENT_CREATED`` event is triggered. In this case, that would be every time a user enrolls in a course. + +.. note:: Consider using asynchronous tasks to handle the event processing to avoid blocking the main thread and improve performance. Also, make sure to handle exceptions and errors gracefully to avoid silent failures and improve debugging. You should also consider not creating a tight coupling between receivers and other services, if doing so is necessary consider using the event bus to broadcast the event. + +Step 5: Test the Event Receiver +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Given the design of Open edX Events, you can include the events definitions in your test suite to ensure that the event receiver is working as expected. You can use the ``send_event`` method to trigger the event and test the event receiver. Here's an example of how you can test the event receiver: + +.. code-block:: python + + from openedx_events import send_event, COURSE_ENROLLMENT_CREATED + + def test_create_notification_preference(): + # Trigger the event + COURSE_ENROLLMENT_CREATED.connect(create_notification_preference) + COURSE_ENROLLMENT_CREATED.send_event( + user=UserData( + pii=UserPersonalData( + username='test_username', + email='test_email@example.com', + name='test_name', + ), + id=1, + is_active=True, + ), + ) + + # Assert that the notification preference was created + assert NotificationPreference.objects.filter(user=enrollment.user).exists() + +This way you can ensure that the event receiver is working as expected and that the custom logic is executed when the event is triggered. If the event definition or payload changes in any way, you can catch the error in the test suite instead of in production. diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst new file mode 100644 index 00000000..380c6af7 --- /dev/null +++ b/docs/how-tos/create-a-new-event.rst @@ -0,0 +1,224 @@ +Create a New Open edX Event with Long-Term Support +================================================== + +This guide describes how to create a new Open edX event with long-term support by following the practices outlined in the :doc:`../decisions/0016-event-design-practices` ADR. + +Events design with long-support follow closely the practices described in the ADR to minimize breaking changes, maximize compatibility and support for future versions of Open edX. + +.. note:: Before starting, ensure you've reviewed the documentation on :doc:`docs.openedx.org:developers/concepts/hooks_extension_framework`, this documentation helps you decide if creating a new event is necessary. You should also review the documentation on :doc:`../decisions/0016-event-design-practices` to understand the practices that should be followed when creating a new event. + +Throughout this guide, we will use an example of creating a new event that will be triggered when a user enrolls in a course from the course about page to better illustrate the steps involved in creating a new event. + +Key Outlines from Event Design Practices +---------------------------------------- + +The :doc:`../decisions/0016-event-design-practices` outlines the following key practices to follow when creating a new event: + +- Clearly describe what happened and why. +- Self-descriptive and self-contained as much as possible. +- Avoid runtime dependencies with other services. +- Avoid ambiguous data fields or fields with multiple meaning. +- Appropriate types and formats. +- Events should have a single responsibility. +- Avoid combining multiple events into one. +- Maintain the right granularity: not too fine-grained or too coarse. +- Ensure the triggering logic is consistent and narrow. +- Keep the event size small. +- Avoid flow control information or business logic in the event. +- Consider the consumers' needs when designing the event. +- Avoid breaking changes. + +Step 1: Propose the Use Case to the Community +---------------------------------------------- + +Before contributing a new event, it is important to propose the event to the community to get feedback on the event's design and use case. For instance, you could create a post in Open edX Discuss Forum or create a new issue in the repository's issue tracker describing your use case for the new event. Here are some examples of community members that have taken this step: + +- `Add Extensibility Mechanism to IDV to Enable Integration of New IDV Vendor Persona`_ +- `Add Program Certificate events`_ + +.. note:: If your use case is too specific to your organization, you can implement them in your own library and use it within your services by adopting an organization-scoped approach leveraging the Apache 2.0 license. However, if you think that your use case could be beneficial to the community, you should propose it to the community for feedback and collaboration. + +In our example our use case proposal could be: + + I want to add an event that will be triggered when a user enrolls in a course from the course about page. This event will be useful for services that need to send notifications to the user when they enroll in a course. + +If you are confident that the event is beneficial to the community, you can proceed to the next steps and implement the event. + +Step 2: Place Your Event In an Architecture Subdomain +----------------------------------------------------- + +To implement the new event in the library, you should understand the purpose of the event and where it fits in the Open edX main architecture subdomains. This will help you place the event in the right architecture subdomain and ensure that the event is consistent with the framework's definitions. Fore more details on the Open edX Architectural Subdomains, refer to the :doc:`../reference/architecture-subdomains`. + +In our example, the event is related to the enrollment process, which is part of the ``learning`` subdomain. Therefore, the event should be placed in the ``/learning`` module in the library. The subdomain is also used as part of the :term:`event type `, which is used to identify the event. The event type should be unique and follow the naming convention for event types specified in the :doc:`../decisions/0002-events-naming-and-versioning` ADR. + +For the enrollment event, the event type could be ``org.openedx.learning.course.enrollment.v1``, where ``learning`` is the subdomain. + +.. note:: If you don't find a suitable subdomain for your event, you can propose a new subdomain to the community. However, new subdomains may require some discussion with the community. So we encourage you to start the conversation as soon as possible through any of the communication channels available. + +Step 3: Determine the Content of the Event +------------------------------------------- + +The content of the event should comply with the practices outlined in the :doc:`../decisions/0016-event-design-practices`. The event should be self-descriptive and self-contained as much as possible. The event should contain all the necessary information for consumers to react to the event without having to make additional calls to other services when possible. + +When determining the content of the event, consider the following: + +- What happened and why? +- What data is needed to describe the event? +- What data is needed to react to the event? + +In our specific example of the enrollment event this could be: + +- What happened: A user enrolled in a course. +- Why: The user enrolled in the course from the course about page. +- Data needed to describe the event: User information (who), course information (where), enrollment date and mode (output details). +- Data needed to react to the event: User information, course information, enrollment Date, enrollment Mode. For instance, a notification could send a welcome email to the user. + +As a rule of thumb, the event should contain the minimum amount of data required to describe the event and react to it. Try including data about each entity involved such that: + +- Consumers can identify the entities involved in the event. +- Key data about the entities is included in the event. +- The outcome of the event is clear. + +This will ensure that the event is self-descriptive and self-contained as much as possible. + +There has been cases where events also carry other contextual data not directly related to the event but useful for consumers. Although this is not recommended, if you need to include such data, ensure that the reasoning behind it is documented and does not introduce ambiguity. + +Step 4: Identify the Event Triggering Logic +------------------------------------------- + +The triggering logic for the event should be identified to ensure that the event is triggered in the right places and that the event is triggered consistently. We should identify the triggering logic to ensure that maximum coverage is achieved with minimal modifications. The goal is to focus on core, critical areas where the logic we want to modify executes, ensuring the event is triggered consistently. + +In our example, the triggering logic could be a place where all enrollment logic goes through. This could be the ``enroll`` method in the enrollment model in the LMS, which is called when a user enrolls in a course in all cases. + +Step 5: Write the Event Definition and Payload +---------------------------------------------- + +Implement the :term:`Event Definition` and :term:`Event Payload` for your event in the corresponding subdomain module. The event definition would be a signal that is triggered when the event takes place, and the event payload would be the data that is included in the event. + +.. note:: Ideally, the data that is included in the event payload should be available at the time the event is triggered, and it should be directly related to the event that took place. So before defining the payload, inspect the triggering logic to review the data that is available at the time the event is triggered. + +The event definition and payload must comply with the practices outlined in the :doc:`../decisions/0002-events-naming-and-versioning` and :doc:`../decisions/0003-events-payload` ADRs. Also, with the practices outlined in the :doc:`../decisions/0016-event-design-practices` ADR. Mainly: + +- The event should be self-descriptive and self-contained as much as possible. +- The event should contain all the necessary information directly related to the event that took place. + +Event Payload +~~~~~~~~~~~~~ + +The event payload is a data `attrs`_ class which defines the data that is included in the event that is defined in the corresponding subdomain module in the ``data.py`` file. The payload should contain all the necessary information directly related to the event that took place to ensure that consumers can react to the event without introducing new dependencies to understand the event. + +In our example, the event definition and payload for the enrollment event could be ``CourseEnrollmentData``. This class should contain all the necessary information about the enrollment event, such as user information, course information, enrollment mode, and other relevant data. + +.. code-block:: python + + # Location openedx_events/learning/data.py + @attr.s(frozen=True) + class CourseEnrollmentData: + """ + Attributes defined for Open edX Course Enrollment object. + + Arguments: + user (UserData): user associated with the Course Enrollment. + course (CourseData): course where the user is enrolled in. + mode (str): course mode associated with the course. + is_active (bool): whether the enrollment is active. + creation_date (datetime): creation date of the enrollment. + created_by (UserData): if available, who created the enrollment. + """ + + user = attr.ib(type=UserData) + course = attr.ib(type=CourseData) + mode = attr.ib(type=str) + is_active = attr.ib(type=bool) + creation_date = attr.ib(type=datetime) + created_by = attr.ib(type=UserData, default=None) + +.. note:: Try grouping the data into logical groups to make the event more readable and maintainable. For instance, in the above example, we have grouped the data into User, Course, and Enrollment data. + +Each field in the payload should be documented with a description of what the field represents and the data type it should contain. This will help consumers understand the payload and react to the event. You should be able to justify why each field is included in the payload and how it relates to the event. + +Event Definition +~~~~~~~~~~~~~~~~ + +The event definition should be defined in the corresponding subdomain module in the ``signals.py`` file. The :term:`Event Definition` should comply with: + +- It must be documented using in-line documentation with at least: ``event_type``, ``event_name``, ``event_description`` and ``event_data``. See :doc:`../reference/in-line-code-annotations-for-an-event` for more information. + +In our example, the event definition for the enrollment event could be: + +.. code-block:: python + + # Location openedx_events/learning/signals.py + # .. event_type: org.openedx.learning.course.enrollment.created.v1 + # .. event_name: COURSE_ENROLLMENT_CREATED + # .. event_description: emitted when the user's enrollment process is completed. + # .. event_data: CourseEnrollmentData + COURSE_ENROLLMENT_CREATED = OpenEdxPublicSignal( + event_type="org.openedx.learning.course.enrollment.created.v1", + data={ + "enrollment": CourseEnrollmentData, + } + ) + +Consumers will be able to access the event payload in their receivers to react to the event. The ``event_type`` is mainly used to identify the event. + +.. TODO: add reference to how to add event bus support to the event's payload + +Step 6: Send the Event +---------------------- + +After defining the event, you should trigger the event in the places we identified in the triggering logic. In our example, we identified that the event should be triggered when a user enrolls in a course so it should be triggered when the enrollment process completes successfully independent of the method of enrollment used. Therefore, we should trigger the event in the ``enroll`` method in the enrollment model in the LMS services. + +Here is how the integration could look like: + +.. code-block:: python + + # Location openedx/core/djangoapps/enrollments/models.py + from openedx_events.learning.signals import COURSE_ENROLLMENT_CREATED + + def enroll(cls, user, course_key, mode=None, **kwargs): + """ + Enroll a user in this course. + """ + # Enrollment logic here + ... + # .. event_implemented_name: COURSE_ENROLLMENT_CREATED + COURSE_ENROLLMENT_CREATED.send_event( + enrollment=CourseEnrollmentData( + user=UserData( + pii=UserPersonalData( + username=user.username, + email=user.email, + name=user.profile.name, + ), + id=user.id, + is_active=user.is_active, + ), + course=course_data, + mode=enrollment.mode, + is_active=enrollment.is_active, + creation_date=enrollment.created, + ) + ) + +.. note:: Ensure that the event is triggered consistently and only when the event should be triggered. Avoid triggering the event multiple times for the same event unless necessary, e.g., when there is no other way to ensure that the event is triggered consistently. + +Step 7: Test the Event +---------------------- + +You should test the event to ensure it triggers consistently and that its payload contains the necessary information. Add unit tests in the service that triggers the event. The main goal is to verify that the event triggers as needed, consumers can react to it, and it carries the expected information. + +In our example, we should add checks for all places where the event is triggered to ensure that the event is triggered consistently. We should also verify that the payload contains the necessary information for consumers to react to the event like user information, course information, enrollment mode, and other relevant data. + +Another way we suggest you test your events is by consuming them in a test environment. This will help you verify that the event is triggered and that the payload contains the necessary information. You can use follow the steps in :doc:`../how-tos/consume-an-event` to consume the event in a test environment with a Django Signal Receiver. Or you could also use the Open edX Event Bus to consume the event in a test environment. For more information on how to use the Open edX Event Bus, refer to the :doc:`../how-tos/use-the-event-bus-to-broadcast-and-consume-events`. + +Step 8: Continue the Contribution Process +----------------------------------------- + +After implementing the event, you should continue the contribution process by creating a pull request in the repository. The pull request should contain the changes you made to implement the event, including the event definition, payload, and the places where the event is triggered. + +For more details on how the contribution flow works, refer to the :doc:`docs.openedx.org:developers/concepts/hooks_extension_framework` documentation. + +.. _Add Extensibility Mechanism to IDV to Enable Integration of New IDV Vendor Persona: https://openedx.atlassian.net/wiki/spaces/OEPM/pages/4307386369/Proposal+Add+Extensibility+Mechanisms+to+IDV+to+Enable+Integration+of+New+IDV+Vendor+Persona +.. _Add Program Certificate events: https://github.com/openedx/openedx-events/issues/250 +.. _attrs: https://www.attrs.org/en/stable/ diff --git a/docs/how-tos/use-the-event-bus-to-broadcast-and-consume-events.rst b/docs/how-tos/use-the-event-bus-to-broadcast-and-consume-events.rst new file mode 100644 index 00000000..f7823d89 --- /dev/null +++ b/docs/how-tos/use-the-event-bus-to-broadcast-and-consume-events.rst @@ -0,0 +1,87 @@ +Use the Open edX Event Bus to Broadcast and Consume Events +========================================================== + +After creating a new Open edX Event, you might need to send it across services instead of just within the same process. For this kind of use-cases, you might want to use the Open edX Event Bus. Here :doc:`../concepts/event-bus`, you can find useful information to start getting familiar with the Open edX Event Bus. + +The Open edX Event Bus is a key component of the Open edX architecture, enabling services to communicate without direct dependencies on each other. This guide provides an overview of how to use the event bus to broadcast Open edX Events to multiple services, allowing them to react to changes or actions in the system. + +Setup +----- + +To start producing and consuming events using the Open edX Event Bus, follow these steps: + +Step 1: Install the Open edX Event Bus Plugin +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +First, you need to install the Open edX Event Bus plugin in both the producing and consuming services. The plugin is a Django app that provides the necessary tools and configurations to produce and consume events. You could install the Redis plugin by running: + +.. code-block:: bash + + pip install edx-event-bus-redis + +.. note:: There are currently two community-supported concrete implementations of the Open edX Events Bus, Redis (`event-bus-redis`_) and Kafka (`event-bus-kafka`_). Redis is the default plugin for the Open edX Event Bus, but you can also use Kafka depending on your requirements. If none of these implementations meet your needs, you can implement your own plugin by following the :doc:`../how-tos/add-new-event-bus-concrete-implementation` documentation. + +Step 2: Configure the Event Bus +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In :doc:`../reference/event-bus-configurations`, you can find the available configurations for the event bus that are used to set up the event bus in the Open edX platform. + +In both the producing and consuming services, you need to configure the event bus. This includes setting up the :term:`event types `, :term:`topics `, and other configurations for the :term:`Event Bus` to work with. In this step, you should configure the producer and consumer classes for the event bus implementation you are using: + +- In the :term:`producing ` service, configure the producer class by setting the ``EVENT_BUS_PRODUCER`` setting. E.g., ``edx_event_bus_redis.create_producer``. +- In the :term:`consuming ` service, configure the consumer class by setting the ``EVENT_BUS_CONSUMER`` setting. E.g., ``edx_event_bus_redis.RedisEventConsumer``. + +By configuring these settings, you are telling Open edX Events which concrete implementation to use for producing and consuming events. + +Step 3: Produce the Event +~~~~~~~~~~~~~~~~~~~~~~~~~ + +In the producing/host application, include ``openedx_events`` in ``INSTALLED_APPS`` settings if necessary and add ``EVENT_BUS_PRODUCER_CONFIG`` setting. This setting is a dictionary of :term:`event types ` to dictionaries for :term:`Topic` related configuration. Each :term:`Topic` configuration dictionary uses the topic as a key and contains: + +- A flag called ``enabled`` denoting whether the event will be published. +- The ``event_key_field`` which is a period-delimited string path to event data field to use as event key. + +.. note:: The topic names should not include environment prefix as it will be dynamically added based on ``EVENT_BUS_TOPIC_PREFIX`` setting. + +Here's an example of the producer configuration which will publish events for XBlock published and deleted events to the specified :term:`Topic`: + +.. code-block:: python + + EVENT_BUS_PRODUCER_CONFIG = { + 'org.openedx.content_authoring.xblock.published.v1': { + 'content-authoring-xblock-lifecycle': {'event_key_field': 'xblock_info.usage_key', 'enabled': True}, + 'content-authoring-xblock-published': {'event_key_field': 'xblock_info.usage_key', 'enabled': True} + }, + 'org.openedx.content_authoring.xblock.deleted.v1': { + 'content-authoring-xblock-lifecycle': {'event_key_field': 'xblock_info.usage_key', 'enabled': True}, + }, + } + +The ``EVENT_BUS_PRODUCER_CONFIG`` is read by ``openedx_events`` and a handler (`general_signal_handler`_) is attached which does the leg work of reading the configuration again and pushing to appropriate handlers. + +Step 4: Consume the Event +~~~~~~~~~~~~~~~~~~~~~~~~~ + +In the consuming service, include ``openedx_events`` in ``INSTALLED_APPS`` settings if necessary and add ``EVENT_BUS_CONSUMER_CONFIG`` setting. Then, you should implement a receiver for the event type you are interested in. In this example, we are interested in the XBlock deleted event: + +.. code-block:: python + + @receiver(XBLOCK_DELETED) + def update_some_data(sender, **kwargs): + ... do things with the data in kwargs ... + ... log the event for debugging purposes ... + +Step 5: Run the Consumer +~~~~~~~~~~~~~~~~~~~~~~~~ + +To consume events, Open edX Events provides a management command called `consume_events`_ which can be called from the command line, how to run this command will depend on your deployment strategy. This command will start a process that listens to the message broker for new messages, processes them and emits the event. Here is an example using of a `consumer using Tutor hosted in Kubernetes`_. + +You can find more a concrete example of how to produce and consume events in the `event-bus-redis`_ documentation. + +.. _consume_events: https://github.com/openedx/openedx-events/blob/main/openedx_events/management/commands/consume_events.py +.. _event-bus-redis: https://github.com/openedx/event-bus-redis +.. _event-bus-kafka: https://github.com/openedx/event-bus-kafka +.. _run the consumer locally without tutor: https://github.com/openedx/event-bus-redis/?tab=readme-ov-file#testing-locally +.. _run the consumer locally with tutor: https://github.com/openedx/event-bus-redis/blob/main/docs/tutor_installation.rst#setup-example-with-openedx-course-discovery-and-tutor +.. _general_signal_handler: https://github.com/openedx/openedx-events/blob/main/openedx_events/apps.py#L16-L44 +.. _consumer using Tutor hosted in Kubernetes: https://github.com/openedx/tutor-contrib-aspects/blob/master/tutoraspects/patches/k8s-deployments#L535-L588 From 56d90d5cbadee168eaa52fad55417d84677c6ed0 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Fri, 20 Dec 2024 10:39:20 +0100 Subject: [PATCH 03/38] docs: add assumptions for how-tos --- docs/how-tos/consume-an-event.rst | 16 +++++++- docs/how-tos/create-a-new-event.rst | 38 +++++++++++++------ ...nt-bus-to-broadcast-and-consume-events.rst | 12 +++++- 3 files changed, 52 insertions(+), 14 deletions(-) diff --git a/docs/how-tos/consume-an-event.rst b/docs/how-tos/consume-an-event.rst index 47abd56d..6d5320c9 100644 --- a/docs/how-tos/consume-an-event.rst +++ b/docs/how-tos/consume-an-event.rst @@ -7,7 +7,16 @@ You have two ways of consuming an Open edX event, within the same service or in Throughout this guide, we will use an example of creating an event handler that will execute when a user enrolls in a course from the course about page to better illustrate the steps involved in creating a consumer for an event. -Setup +Assumptions +----------- + +- You have a development environment set up using `Tutor`_. +- You have a basic understanding of Python and Django. +- You have created a new Open edX event. If not, you can follow the :doc:`../how-tos/create-a-new-event` guide to create a new event. +- You have a basic understanding of Django signals. If not, you can review the `Django Signals Documentation`_. +- You are familiar with the terminology used in the project, such as the terms :term:`Event Type` or :term:`Event Receiver`. If not, you can review the :doc:`../reference/glossary` documentation. + +Steps ----- To consume an event within the same service, follow these steps: @@ -42,7 +51,7 @@ Now, the django dispatcher will call the ``create_notification_preference`` func .. note:: Consider using asynchronous tasks to handle the event processing to avoid blocking the main thread and improve performance. Also, make sure to handle exceptions and errors gracefully to avoid silent failures and improve debugging. You should also consider not creating a tight coupling between receivers and other services, if doing so is necessary consider using the event bus to broadcast the event. -Step 5: Test the Event Receiver +Step 3: Test the Event Receiver ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Given the design of Open edX Events, you can include the events definitions in your test suite to ensure that the event receiver is working as expected. You can use the ``send_event`` method to trigger the event and test the event receiver. Here's an example of how you can test the event receiver: @@ -70,3 +79,6 @@ Given the design of Open edX Events, you can include the events definitions in y assert NotificationPreference.objects.filter(user=enrollment.user).exists() This way you can ensure that the event receiver is working as expected and that the custom logic is executed when the event is triggered. If the event definition or payload changes in any way, you can catch the error in the test suite instead of in production. + +.. _Tutor: https://docs.tutor.edly.io/ +.. _Django Signals Documentation: https://docs.djangoproject.com/en/4.2/topics/signals/ diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index 380c6af7..808ee124 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -1,7 +1,7 @@ Create a New Open edX Event with Long-Term Support ================================================== -This guide describes how to create a new Open edX event with long-term support by following the practices outlined in the :doc:`../decisions/0016-event-design-practices` ADR. +Open edX Events are supported and maintained by the Open edX community. This mechanism is designed to be extensible and flexible to allow developers to create new events that can be consumed by other services. This guide describes how to create a new Open edX event with long-term support by following the practices outlined in the :doc:`../decisions/0016-event-design-practices` ADR. Events design with long-support follow closely the practices described in the ADR to minimize breaking changes, maximize compatibility and support for future versions of Open edX. @@ -28,8 +28,22 @@ The :doc:`../decisions/0016-event-design-practices` outlines the following key p - Consider the consumers' needs when designing the event. - Avoid breaking changes. +Assumptions +----------- + +- You have a development environment set up using `Tutor`_. +- You have a basic understanding of Python and Django. +- You have a basic understanding of Django signals. If not, you can review the `Django Signals Documentation`_. +- You understand the concept of events or have reviewed the relevant :doc:`/concepts/index` docs. +- You are familiar with the terminology used in the project, such as the terms :term:`Event Type` or :term:`Event Payload`. If not, you can review the :doc:`/reference/glossary` docs. + +Steps +----- + +To create a new Open edX event with long-term support, follow these steps: + Step 1: Propose the Use Case to the Community ----------------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Before contributing a new event, it is important to propose the event to the community to get feedback on the event's design and use case. For instance, you could create a post in Open edX Discuss Forum or create a new issue in the repository's issue tracker describing your use case for the new event. Here are some examples of community members that have taken this step: @@ -45,7 +59,7 @@ In our example our use case proposal could be: If you are confident that the event is beneficial to the community, you can proceed to the next steps and implement the event. Step 2: Place Your Event In an Architecture Subdomain ------------------------------------------------------ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To implement the new event in the library, you should understand the purpose of the event and where it fits in the Open edX main architecture subdomains. This will help you place the event in the right architecture subdomain and ensure that the event is consistent with the framework's definitions. Fore more details on the Open edX Architectural Subdomains, refer to the :doc:`../reference/architecture-subdomains`. @@ -56,7 +70,7 @@ For the enrollment event, the event type could be ``org.openedx.learning.course. .. note:: If you don't find a suitable subdomain for your event, you can propose a new subdomain to the community. However, new subdomains may require some discussion with the community. So we encourage you to start the conversation as soon as possible through any of the communication channels available. Step 3: Determine the Content of the Event -------------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The content of the event should comply with the practices outlined in the :doc:`../decisions/0016-event-design-practices`. The event should be self-descriptive and self-contained as much as possible. The event should contain all the necessary information for consumers to react to the event without having to make additional calls to other services when possible. @@ -84,14 +98,14 @@ This will ensure that the event is self-descriptive and self-contained as much a There has been cases where events also carry other contextual data not directly related to the event but useful for consumers. Although this is not recommended, if you need to include such data, ensure that the reasoning behind it is documented and does not introduce ambiguity. Step 4: Identify the Event Triggering Logic -------------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The triggering logic for the event should be identified to ensure that the event is triggered in the right places and that the event is triggered consistently. We should identify the triggering logic to ensure that maximum coverage is achieved with minimal modifications. The goal is to focus on core, critical areas where the logic we want to modify executes, ensuring the event is triggered consistently. In our example, the triggering logic could be a place where all enrollment logic goes through. This could be the ``enroll`` method in the enrollment model in the LMS, which is called when a user enrolls in a course in all cases. Step 5: Write the Event Definition and Payload ----------------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Implement the :term:`Event Definition` and :term:`Event Payload` for your event in the corresponding subdomain module. The event definition would be a signal that is triggered when the event takes place, and the event payload would be the data that is included in the event. @@ -103,7 +117,7 @@ The event definition and payload must comply with the practices outlined in the - The event should contain all the necessary information directly related to the event that took place. Event Payload -~~~~~~~~~~~~~ +************* The event payload is a data `attrs`_ class which defines the data that is included in the event that is defined in the corresponding subdomain module in the ``data.py`` file. The payload should contain all the necessary information directly related to the event that took place to ensure that consumers can react to the event without introducing new dependencies to understand the event. @@ -138,7 +152,7 @@ In our example, the event definition and payload for the enrollment event could Each field in the payload should be documented with a description of what the field represents and the data type it should contain. This will help consumers understand the payload and react to the event. You should be able to justify why each field is included in the payload and how it relates to the event. Event Definition -~~~~~~~~~~~~~~~~ +**************** The event definition should be defined in the corresponding subdomain module in the ``signals.py`` file. The :term:`Event Definition` should comply with: @@ -165,7 +179,7 @@ Consumers will be able to access the event payload in their receivers to react t .. TODO: add reference to how to add event bus support to the event's payload Step 6: Send the Event ----------------------- +~~~~~~~~~~~~~~~~~~~~~~~ After defining the event, you should trigger the event in the places we identified in the triggering logic. In our example, we identified that the event should be triggered when a user enrolls in a course so it should be triggered when the enrollment process completes successfully independent of the method of enrollment used. Therefore, we should trigger the event in the ``enroll`` method in the enrollment model in the LMS services. @@ -204,7 +218,7 @@ Here is how the integration could look like: .. note:: Ensure that the event is triggered consistently and only when the event should be triggered. Avoid triggering the event multiple times for the same event unless necessary, e.g., when there is no other way to ensure that the event is triggered consistently. Step 7: Test the Event ----------------------- +~~~~~~~~~~~~~~~~~~~~~~~ You should test the event to ensure it triggers consistently and that its payload contains the necessary information. Add unit tests in the service that triggers the event. The main goal is to verify that the event triggers as needed, consumers can react to it, and it carries the expected information. @@ -213,7 +227,7 @@ In our example, we should add checks for all places where the event is triggered Another way we suggest you test your events is by consuming them in a test environment. This will help you verify that the event is triggered and that the payload contains the necessary information. You can use follow the steps in :doc:`../how-tos/consume-an-event` to consume the event in a test environment with a Django Signal Receiver. Or you could also use the Open edX Event Bus to consume the event in a test environment. For more information on how to use the Open edX Event Bus, refer to the :doc:`../how-tos/use-the-event-bus-to-broadcast-and-consume-events`. Step 8: Continue the Contribution Process ------------------------------------------ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ After implementing the event, you should continue the contribution process by creating a pull request in the repository. The pull request should contain the changes you made to implement the event, including the event definition, payload, and the places where the event is triggered. @@ -222,3 +236,5 @@ For more details on how the contribution flow works, refer to the :doc:`docs.ope .. _Add Extensibility Mechanism to IDV to Enable Integration of New IDV Vendor Persona: https://openedx.atlassian.net/wiki/spaces/OEPM/pages/4307386369/Proposal+Add+Extensibility+Mechanisms+to+IDV+to+Enable+Integration+of+New+IDV+Vendor+Persona .. _Add Program Certificate events: https://github.com/openedx/openedx-events/issues/250 .. _attrs: https://www.attrs.org/en/stable/ +.. _Tutor: https://docs.tutor.edly.io/ +.. _Django Signals Documentation: https://docs.djangoproject.com/en/4.2/topics/signals/ diff --git a/docs/how-tos/use-the-event-bus-to-broadcast-and-consume-events.rst b/docs/how-tos/use-the-event-bus-to-broadcast-and-consume-events.rst index f7823d89..92985f7f 100644 --- a/docs/how-tos/use-the-event-bus-to-broadcast-and-consume-events.rst +++ b/docs/how-tos/use-the-event-bus-to-broadcast-and-consume-events.rst @@ -5,7 +5,16 @@ After creating a new Open edX Event, you might need to send it across services i The Open edX Event Bus is a key component of the Open edX architecture, enabling services to communicate without direct dependencies on each other. This guide provides an overview of how to use the event bus to broadcast Open edX Events to multiple services, allowing them to react to changes or actions in the system. -Setup +Assumptions +----------- + +- You have a development environment set up using `Tutor`_. +- You have a basic understanding of Python and Django. +- You have basic understanding of the Open edX Event Bus. If not, you can review the :doc:`../concepts/event-bus` docs. +- You understand the concept of filters or have reviewed the relevant :doc:`/concepts/index` docs. +- You are familiar with the terminology used in the project, such as the terms :term:`Event Type` or :term:`Topic`. If not, you can review the :doc:`/reference/glossary` docs. + +Steps ----- To start producing and consuming events using the Open edX Event Bus, follow these steps: @@ -85,3 +94,4 @@ You can find more a concrete example of how to produce and consume events in the .. _run the consumer locally with tutor: https://github.com/openedx/event-bus-redis/blob/main/docs/tutor_installation.rst#setup-example-with-openedx-course-discovery-and-tutor .. _general_signal_handler: https://github.com/openedx/openedx-events/blob/main/openedx_events/apps.py#L16-L44 .. _consumer using Tutor hosted in Kubernetes: https://github.com/openedx/tutor-contrib-aspects/blob/master/tutoraspects/patches/k8s-deployments#L535-L588 +.. _Tutor: https://docs.tutor.edly.io/ From 155f177ca17195f40b0513922de74b440301c0f2 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Fri, 20 Dec 2024 10:50:00 +0100 Subject: [PATCH 04/38] docs: add more considerations to keep in mind --- docs/how-tos/create-a-new-event.rst | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index 808ee124..df56d723 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -36,6 +36,8 @@ Assumptions - You have a basic understanding of Django signals. If not, you can review the `Django Signals Documentation`_. - You understand the concept of events or have reviewed the relevant :doc:`/concepts/index` docs. - You are familiar with the terminology used in the project, such as the terms :term:`Event Type` or :term:`Event Payload`. If not, you can review the :doc:`/reference/glossary` docs. +- You have reviewed the :doc:`../decisions/0016-event-design-practices` ADR. +- You have identified that you need to create a new event and have a use case for the event. Steps ----- @@ -93,9 +95,9 @@ As a rule of thumb, the event should contain the minimum amount of data required - Key data about the entities is included in the event. - The outcome of the event is clear. -This will ensure that the event is self-descriptive and self-contained as much as possible. +This will help ensure that the event is self-descriptive and self-contained as much as possible. -There has been cases where events also carry other contextual data not directly related to the event but useful for consumers. Although this is not recommended, if you need to include such data, ensure that the reasoning behind it is documented and does not introduce ambiguity. +.. note:: There has been cases where events also carry other contextual data not directly related to the event but useful for consumers. Although this is not recommended, if you need to include such data, ensure that the reasoning behind it is documented and does not introduce ambiguity. Step 4: Identify the Event Triggering Logic ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -115,6 +117,7 @@ The event definition and payload must comply with the practices outlined in the - The event should be self-descriptive and self-contained as much as possible. - The event should contain all the necessary information directly related to the event that took place. +- Maintain the right granularity: not too fine-grained or too coarse to ensure that the event is useful for consumers. Event Payload ************* @@ -151,6 +154,8 @@ In our example, the event definition and payload for the enrollment event could Each field in the payload should be documented with a description of what the field represents and the data type it should contain. This will help consumers understand the payload and react to the event. You should be able to justify why each field is included in the payload and how it relates to the event. +.. note:: Try reusing existing data classes if possible to avoid duplicating data classes. This will help maintain consistency and reduce the chances of introducing errors. + Event Definition **************** From fb047b08192c32abddd40a6890a6820f7d613ee8 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Fri, 20 Dec 2024 12:45:01 +0100 Subject: [PATCH 05/38] docs: add consequences for ADR --- docs/decisions/0016-event-design-practices.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/decisions/0016-event-design-practices.rst b/docs/decisions/0016-event-design-practices.rst index dba0dd2c..ab8e466b 100644 --- a/docs/decisions/0016-event-design-practices.rst +++ b/docs/decisions/0016-event-design-practices.rst @@ -125,6 +125,11 @@ Some of these practices might not be applicable to all events, but they are a go In addition to these practices, review the Architectural Decision Records (ADRs) related to events to understand the naming, versioning, payload, and other practices that are specific to Open edX events. +Consequences +------------ + +Following these practices will help ensure that the events are consistent, maintainable, and reusable. It will also help consumers understand the message and react to the event accordingly. However, it might require additional effort to design the event and ensure that it contains the necessary information for consumers to react to the event, although this effort will pay off in the long run. Having these standards in place will also make the decision process easier when designing new events. + .. _Event-Driven Microservices: https://www.oreilly.com/library/view/building-event-driven-microservices/9781492057888/ .. _Event-Driven article: https://martinfowler.com/articles/201701-event-driven.html .. _Thin Events - The lean muscle of event-driven architecture: https://www.thoughtworks.com/insights/blog/architecture/thin-events-the-lean-muscle-of-event-driven-architecture From 3721ecbb27a429399648cb6a967a9ad726f47e8c Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Fri, 20 Dec 2024 12:51:59 +0100 Subject: [PATCH 06/38] docs: add note about consumption --- docs/decisions/0016-event-design-practices.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/decisions/0016-event-design-practices.rst b/docs/decisions/0016-event-design-practices.rst index ab8e466b..a0fbeda1 100644 --- a/docs/decisions/0016-event-design-practices.rst +++ b/docs/decisions/0016-event-design-practices.rst @@ -26,7 +26,7 @@ Event Purpose and Content ~~~~~~~~~~~~~~~~~~~~~~~~~ - An event should describe as accurately as possible what happened (what) and why it happened (why). It must contain enough information for consumers to understand the message. For instance, if an event is about a user enrollment, it should contain the user's data, the course data, and the enrollment status and the event should be named accordingly. -- Avoid immediately contacting the source service to retrieve additional information. Instead, consider adding the necessary information to the event payload by managing the granularity of the event. If the event requires additional information, consider adding a field to the event that contains the necessary information. This will reduce the number of dependencies between services and make the event more self-contained. +- Avoid immediately contacting the source service to retrieve additional information from the consumer-side. Instead, consider adding the necessary information to the event payload by managing the granularity of the event. If the event requires additional information, consider adding a field to the event that contains the necessary information. This will reduce the number of dependencies between services and make the event more self-contained. - Keep the event size small. Avoid adding unnecessary information to the event. If the information is not necessary for consumers to react to the event, consider removing it. - Avoid adding flow-control information or business logic to events. Events should be solely a representation of what took place. If a field is necessary to control the behavior of the consumer, consider moving it to the consumer side. If adding additional data to the event is absolutely necessary document the reasoning behind it and carefully study the use case and implications. From af8493ac078b4d1b388ccc886d45b757f1e8ab0c Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Fri, 20 Dec 2024 14:50:54 +0100 Subject: [PATCH 07/38] docs: add note about considering support over time for the service --- docs/how-tos/create-a-new-event.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index df56d723..7e29c0a4 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -106,6 +106,8 @@ The triggering logic for the event should be identified to ensure that the event In our example, the triggering logic could be a place where all enrollment logic goes through. This could be the ``enroll`` method in the enrollment model in the LMS, which is called when a user enrolls in a course in all cases. +.. note:: When designing an event take into account the support over time of the service and triggering logic. If the service is likely to change or be deprecated, consider the implications of implementing the event in that service. + Step 5: Write the Event Definition and Payload ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From fc49b97090687543c476a2e1aac4d020a7c26d91 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Fri, 20 Dec 2024 16:18:19 +0100 Subject: [PATCH 08/38] docs: move identify triggering logic to previous step --- docs/how-tos/create-a-new-event.rst | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index 7e29c0a4..57adb925 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -35,14 +35,14 @@ Assumptions - You have a basic understanding of Python and Django. - You have a basic understanding of Django signals. If not, you can review the `Django Signals Documentation`_. - You understand the concept of events or have reviewed the relevant :doc:`/concepts/index` docs. -- You are familiar with the terminology used in the project, such as the terms :term:`Event Type` or :term:`Event Payload`. If not, you can review the :doc:`/reference/glossary` docs. +- You are familiar with the terminology used in the project, such as the terms :term:`Event Type` or :term:`Event Payload`. If not, you can review the :doc:`../reference/glossary` docs. - You have reviewed the :doc:`../decisions/0016-event-design-practices` ADR. - You have identified that you need to create a new event and have a use case for the event. Steps ----- -To create a new Open edX event with long-term support, follow these steps: +To create a new Open edX Event with long-term support, follow these steps: Step 1: Propose the Use Case to the Community ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -71,7 +71,18 @@ For the enrollment event, the event type could be ``org.openedx.learning.course. .. note:: If you don't find a suitable subdomain for your event, you can propose a new subdomain to the community. However, new subdomains may require some discussion with the community. So we encourage you to start the conversation as soon as possible through any of the communication channels available. -Step 3: Determine the Content of the Event +Step 3: Identify the Event Triggering Logic +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The triggering logic for the event should be identified to ensure that the event is triggered in the right places and that the event is triggered consistently. We should identify the triggering logic to ensure that maximum coverage is achieved with minimal modifications. The goal is to focus on core, critical areas where the logic we want to modify executes, ensuring the event is triggered consistently. + +In our example, the triggering logic could be a place where all enrollment logic goes through. This could be the ``enroll`` method in the enrollment model in the LMS, which is called when a user enrolls in a course in all cases. + +.. note:: When designing an event take into account the support over time of the service and triggering logic. If the service is likely to change or be deprecated, consider the implications of implementing the event in that service. + +.. note:: It is helpful to inspect the triggering logic to review the data that is available at the time the event is triggered. This will help you determine the content of the event and the data that should be included in the event payload. + +Step 4: Determine the Content of the Event ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The content of the event should comply with the practices outlined in the :doc:`../decisions/0016-event-design-practices`. The event should be self-descriptive and self-contained as much as possible. The event should contain all the necessary information for consumers to react to the event without having to make additional calls to other services when possible. @@ -99,15 +110,6 @@ This will help ensure that the event is self-descriptive and self-contained as m .. note:: There has been cases where events also carry other contextual data not directly related to the event but useful for consumers. Although this is not recommended, if you need to include such data, ensure that the reasoning behind it is documented and does not introduce ambiguity. -Step 4: Identify the Event Triggering Logic -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The triggering logic for the event should be identified to ensure that the event is triggered in the right places and that the event is triggered consistently. We should identify the triggering logic to ensure that maximum coverage is achieved with minimal modifications. The goal is to focus on core, critical areas where the logic we want to modify executes, ensuring the event is triggered consistently. - -In our example, the triggering logic could be a place where all enrollment logic goes through. This could be the ``enroll`` method in the enrollment model in the LMS, which is called when a user enrolls in a course in all cases. - -.. note:: When designing an event take into account the support over time of the service and triggering logic. If the service is likely to change or be deprecated, consider the implications of implementing the event in that service. - Step 5: Write the Event Definition and Payload ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From c10618a484796064c3cdc2f6826991662948b41d Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Fri, 20 Dec 2024 16:21:51 +0100 Subject: [PATCH 09/38] docs: use implement instead of write --- docs/how-tos/create-a-new-event.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index 57adb925..4c64d7da 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -110,8 +110,8 @@ This will help ensure that the event is self-descriptive and self-contained as m .. note:: There has been cases where events also carry other contextual data not directly related to the event but useful for consumers. Although this is not recommended, if you need to include such data, ensure that the reasoning behind it is documented and does not introduce ambiguity. -Step 5: Write the Event Definition and Payload -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Step 5: Implement the Event Definition and Payload +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Implement the :term:`Event Definition` and :term:`Event Payload` for your event in the corresponding subdomain module. The event definition would be a signal that is triggered when the event takes place, and the event payload would be the data that is included in the event. From 3c8ace2ffe55b7235873a4722e9bd0141c18d3e1 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Fri, 20 Dec 2024 16:53:20 +0100 Subject: [PATCH 10/38] docs: add note about data stability over time --- docs/how-tos/create-a-new-event.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index 4c64d7da..b931d0c0 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -110,6 +110,8 @@ This will help ensure that the event is self-descriptive and self-contained as m .. note:: There has been cases where events also carry other contextual data not directly related to the event but useful for consumers. Although this is not recommended, if you need to include such data, ensure that the reasoning behind it is documented and does not introduce ambiguity. +.. note:: Also consider how relevant is the data to where the event is triggered. Consider whether it could be removed or deprecated in the future so that the event remains consistent and maintainable over time. + Step 5: Implement the Event Definition and Payload ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From 1b477dcd17cd552e8d4ed4f8c57438b04bc726e4 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Fri, 20 Dec 2024 17:02:11 +0100 Subject: [PATCH 11/38] docs: split tests needed for the new event --- docs/how-tos/create-a-new-event.rst | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index b931d0c0..1fa96f0c 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -233,7 +233,11 @@ Step 7: Test the Event You should test the event to ensure it triggers consistently and that its payload contains the necessary information. Add unit tests in the service that triggers the event. The main goal is to verify that the event triggers as needed, consumers can react to it, and it carries the expected information. -In our example, we should add checks for all places where the event is triggered to ensure that the event is triggered consistently. We should also verify that the payload contains the necessary information for consumers to react to the event like user information, course information, enrollment mode, and other relevant data. +To ensure that our example is tested thoroughly, we should: + +- Add unit tests to the ``enroll`` method to ensure that the event is triggered when a user enrolls in a course. This means testing the event is triggered when the enrollment process completes successfully. +- Add checks to ensure that the event is triggered consistently and only when the event should be triggered. +- Verify that the payload contains the necessary information for consumers to react to the event like user information, course information, enrollment mode, and other relevant data. Another way we suggest you test your events is by consuming them in a test environment. This will help you verify that the event is triggered and that the payload contains the necessary information. You can use follow the steps in :doc:`../how-tos/consume-an-event` to consume the event in a test environment with a Django Signal Receiver. Or you could also use the Open edX Event Bus to consume the event in a test environment. For more information on how to use the Open edX Event Bus, refer to the :doc:`../how-tos/use-the-event-bus-to-broadcast-and-consume-events`. From a9b0a1f277aeaab95c806815e31fd3538121e4c0 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Fri, 20 Dec 2024 18:07:49 +0100 Subject: [PATCH 12/38] docs: add note about triggering only when facts happen --- docs/how-tos/create-a-new-event.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index 1fa96f0c..d8a4a366 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -228,6 +228,8 @@ Here is how the integration could look like: .. note:: Ensure that the event is triggered consistently and only when the event should be triggered. Avoid triggering the event multiple times for the same event unless necessary, e.g., when there is no other way to ensure that the event is triggered consistently. +.. note:: Try placing the event after the triggering logic to ensure that the event is triggered only when the triggering logic completes successfully. This will help ensure that the event is triggered only for factual events if the triggering logic fails, the event should not be triggered. + Step 7: Test the Event ~~~~~~~~~~~~~~~~~~~~~~~ From 07a130d5616decc038c3694b7e14fb383aa2fc11 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Fri, 20 Dec 2024 18:59:06 +0100 Subject: [PATCH 13/38] docs: add independent step for consuming the event --- docs/how-tos/create-a-new-event.rst | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index d8a4a366..79af3edb 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -241,9 +241,12 @@ To ensure that our example is tested thoroughly, we should: - Add checks to ensure that the event is triggered consistently and only when the event should be triggered. - Verify that the payload contains the necessary information for consumers to react to the event like user information, course information, enrollment mode, and other relevant data. -Another way we suggest you test your events is by consuming them in a test environment. This will help you verify that the event is triggered and that the payload contains the necessary information. You can use follow the steps in :doc:`../how-tos/consume-an-event` to consume the event in a test environment with a Django Signal Receiver. Or you could also use the Open edX Event Bus to consume the event in a test environment. For more information on how to use the Open edX Event Bus, refer to the :doc:`../how-tos/use-the-event-bus-to-broadcast-and-consume-events`. +Step 8: Consume the Event +~~~~~~~~~~~~~~~~~~~~~~~~~ -Step 8: Continue the Contribution Process +Since the event is now implemented, you should consume the event to verify that it is triggered and that the payload contains the necessary information. You can consume the event in a test environment using a Django Signal Receiver. This will help you verify that the event is triggered and that the payload contains the necessary information. You can use follow the steps in :doc:`../how-tos/consume-an-event` to consume the event in a test environment with a Django Signal Receiver. Or you could also use the Open edX Event Bus to consume the event in a test environment. For more information on how to use the Open edX Event Bus, refer to the :doc:`../how-tos/use-the-event-bus-to-broadcast-and-consume-events`. + +Step 9: Continue the Contribution Process ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ After implementing the event, you should continue the contribution process by creating a pull request in the repository. The pull request should contain the changes you made to implement the event, including the event definition, payload, and the places where the event is triggered. From 7909e31312200e03eab93cf32ed3cca520cd9096 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Fri, 20 Dec 2024 19:25:01 +0100 Subject: [PATCH 14/38] docs: add note about adding tests to the triggering logic --- docs/how-tos/create-a-new-event.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index 79af3edb..406141f1 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -241,6 +241,8 @@ To ensure that our example is tested thoroughly, we should: - Add checks to ensure that the event is triggered consistently and only when the event should be triggered. - Verify that the payload contains the necessary information for consumers to react to the event like user information, course information, enrollment mode, and other relevant data. +There is no need to test the event definition since the tooling already tests the definitions for you, but you should test the event triggering logic to ensure that the event complies with the expected behavior. + Step 8: Consume the Event ~~~~~~~~~~~~~~~~~~~~~~~~~ From 964228ad00f7a77fc793c7b00986606c503a7ae0 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Fri, 20 Dec 2024 19:46:27 +0100 Subject: [PATCH 15/38] docs: add an example of testing integration --- docs/how-tos/create-a-new-event.rst | 58 ++++++++++++++++++++++++++++- 1 file changed, 57 insertions(+), 1 deletion(-) diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index 406141f1..6d9cf9ea 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -198,7 +198,7 @@ Here is how the integration could look like: .. code-block:: python - # Location openedx/core/djangoapps/enrollments/models.py + # Location common/djangoapps/student/models.py from openedx_events.learning.signals import COURSE_ENROLLMENT_CREATED def enroll(cls, user, course_key, mode=None, **kwargs): @@ -243,6 +243,62 @@ To ensure that our example is tested thoroughly, we should: There is no need to test the event definition since the tooling already tests the definitions for you, but you should test the event triggering logic to ensure that the event complies with the expected behavior. +In our example, we could write a test that enrolls a user in a course and verifies that the event is triggered with the correct payload. Here is an example of how the test could look like: + +.. code-block:: python + + # Location common/djangoapps/student/tests/test_events.py + from openedx_events.learning.signals import COURSE_ENROLLMENT_CREATED + + def _event_receiver_side_effect(self, **kwargs): + """ + Used show that the Open edX Event was called by the Django signal handler. + """ + self.receiver_called = True + + def test_enrollment_created_event_emitted(self): + """ + Test whether the student enrollment event is sent after the user's enrollment process. + + Expected result: + - COURSE_ENROLLMENT_CREATED is sent and received by the mocked receiver. + - The arguments that the receiver gets are the arguments sent by the event + except the metadata generated on the fly. + """ + event_receiver = mock.Mock(side_effect=self._event_receiver_side_effect) + COURSE_ENROLLMENT_CREATED.connect(event_receiver) + + enrollment = CourseEnrollment.enroll(self.user, self.course.id) + + self.assertTrue(self.receiver_called) + self.assertDictContainsSubset( + { + "signal": COURSE_ENROLLMENT_CREATED, + "sender": None, + "enrollment": CourseEnrollmentData( + user=UserData( + pii=UserPersonalData( + username=self.user.username, + email=self.user.email, + name=self.user.profile.name, + ), + id=self.user.id, + is_active=self.user.is_active, + ), + course=CourseData( + course_key=self.course.id, + display_name=self.course.display_name, + ), + mode=enrollment.mode, + is_active=enrollment.is_active, + creation_date=enrollment.created, + ), + }, + event_receiver.call_args.kwargs + ) + +.. note:: Ensure that the test verifies that the event is triggered when the enrollment process completes successfully and that the payload contains the necessary information. + Step 8: Consume the Event ~~~~~~~~~~~~~~~~~~~~~~~~~ From 2d791c80cc11ad9f34f5ed00950d4d1bac5d3c13 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Wed, 25 Dec 2024 12:40:33 +0100 Subject: [PATCH 16/38] docs: add more details about example code --- docs/how-tos/consume-an-event.rst | 38 ++++++++++++++++++++--------- docs/how-tos/create-a-new-event.rst | 34 ++++++++++++++------------ 2 files changed, 45 insertions(+), 27 deletions(-) diff --git a/docs/how-tos/consume-an-event.rst b/docs/how-tos/consume-an-event.rst index 6d5320c9..71c4c853 100644 --- a/docs/how-tos/consume-an-event.rst +++ b/docs/how-tos/consume-an-event.rst @@ -47,9 +47,8 @@ An :term:`Event Receiver` is simply a function that listens for a specific event # Custom logic to create a notification preference for the user pass -Now, the django dispatcher will call the ``create_notification_preference`` function when the ``COURSE_ENROLLMENT_CREATED`` event is triggered. In this case, that would be every time a user enrolls in a course. - -.. note:: Consider using asynchronous tasks to handle the event processing to avoid blocking the main thread and improve performance. Also, make sure to handle exceptions and errors gracefully to avoid silent failures and improve debugging. You should also consider not creating a tight coupling between receivers and other services, if doing so is necessary consider using the event bus to broadcast the event. +- The Django dispatcher will call the ``create_notification_preference`` function when the ``COURSE_ENROLLMENT_CREATED`` event is triggered by using the ``receiver`` decorator. In this case, that would be every time a user enrolls in a course. +- Consider using asynchronous tasks to handle the event processing to avoid blocking the main thread and improve performance. Also, make sure to handle exceptions and errors gracefully to avoid silent failures and improve debugging. You should also consider not creating a tight coupling between receivers and other services, if doing so is necessary consider using the event bus to broadcast the event. Step 3: Test the Event Receiver ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -62,21 +61,36 @@ Given the design of Open edX Events, you can include the events definitions in y def test_create_notification_preference(): # Trigger the event - COURSE_ENROLLMENT_CREATED.connect(create_notification_preference) - COURSE_ENROLLMENT_CREATED.send_event( + enrollment_data = CourseEnrollmentData( user=UserData( pii=UserPersonalData( - username='test_username', - email='test_email@example.com', - name='test_name', + username=self.user.username, + email=self.user.email, + name=self.user.profile.name, ), - id=1, - is_active=True, + id=self.user.id, + is_active=self.user.is_active, ), + course=CourseData( + course_key=self.course.id, + display_name=self.course.display_name, + ), + mode=self.course_enrollment.mode, + is_active=self.course_enrollment.is_active, + creation_date=self.course_enrollment.created, ) + COURSE_ENROLLMENT_CREATED.send_event( + enrollment=enrollment_data + ) + + # Assert that CourseNotificationPreference object was created with correct attributes + notification_preferences = CourseNotificationPreference.objects.all() + + self.assertEqual(notification_preferences.count(), 1) + self.assertEqual(notification_preferences[0].user, self.user) - # Assert that the notification preference was created - assert NotificationPreference.objects.filter(user=enrollment.user).exists() +- In the test suite, you can use the ``send_event`` method to trigger the event and pass the necessary data to the event receiver. In this case, we are passing the user, course and enrollment data to the event receiver as the triggering logic would do. +- After triggering the event, you can assert that the event receiver executed the custom logic as expected. In this case, we are checking that a ``CourseNotificationPreference`` object was created with the correct attributes. This way you can ensure that the event receiver is working as expected and that the custom logic is executed when the event is triggered. If the event definition or payload changes in any way, you can catch the error in the test suite instead of in production. diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index 6d9cf9ea..895597d6 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -156,20 +156,17 @@ In our example, the event definition and payload for the enrollment event could creation_date = attr.ib(type=datetime) created_by = attr.ib(type=UserData, default=None) -.. note:: Try grouping the data into logical groups to make the event more readable and maintainable. For instance, in the above example, we have grouped the data into User, Course, and Enrollment data. - -Each field in the payload should be documented with a description of what the field represents and the data type it should contain. This will help consumers understand the payload and react to the event. You should be able to justify why each field is included in the payload and how it relates to the event. - -.. note:: Try reusing existing data classes if possible to avoid duplicating data classes. This will help maintain consistency and reduce the chances of introducing errors. +- The payload should be an `attrs`_ class to ensure that the data is immutable by using the ``frozen=True`` argument and to ensure that the data is self-descriptive. +- Use the ``attr.ib`` decorator to define the fields in the payload with the data type that the field should contain. Try to use the appropriate data type for each field to ensure that the data is consistent and maintainable, you can inspect the triggering logic to review the data that is available at the time the event is triggered. +- Try using nested data classes to group related data together. This will help maintain consistency and make the event more readable. For instance, in the above example, we have grouped the data into User, Course, and Enrollment data. +- Try reusing existing data classes if possible to avoid duplicating data classes. This will help maintain consistency and reduce the chances of introducing errors. +- Each field in the payload should be documented with a description of what the field represents and the data type it should contain. This will help consumers understand the payload and react to the event. You should be able to justify why each field is included in the payload and how it relates to the event. +- Use defaults for optional fields in the payload to ensure its consistency in all cases. Event Definition **************** -The event definition should be defined in the corresponding subdomain module in the ``signals.py`` file. The :term:`Event Definition` should comply with: - -- It must be documented using in-line documentation with at least: ``event_type``, ``event_name``, ``event_description`` and ``event_data``. See :doc:`../reference/in-line-code-annotations-for-an-event` for more information. - -In our example, the event definition for the enrollment event could be: +The :term:`Event Definition` should be implemented in the corresponding subdomain module in the ``signals.py`` file. In our example, the event definition for the enrollment event could be: .. code-block:: python @@ -185,7 +182,13 @@ In our example, the event definition for the enrollment event could be: } ) -Consumers will be able to access the event payload in their receivers to react to the event. The ``event_type`` is mainly used to identify the event. +- The event definition should be documented using in-line documentation with at lest ``event_type``, ``event_name``, ``event_description`` and ``event_data``. This will help consumers understand the event and react to it. See :doc:`../reference/in-line-code-annotations-for-an-event` for more information. +- The :term:`Event Type` should be unique and follow the naming convention for event types specified in the :doc:`../decisions/0002-events-naming-and-versioning` ADR. This is used by consumers to identify the event. +- The ``event_name`` should be a constant that is used to identify the event in the code. +- The ``event_description`` should describe what the event is about and why it is triggered. +- The ``event_data`` should be the payload class that is used to define the data that is included in the event. +- The event should be an instance of the ``OpenEdxPublicSignal`` class to ensure that the event is consistent with the Open edX event framework. +- Receivers should be able to access the event payload in their receivers to react to the event. .. TODO: add reference to how to add event bus support to the event's payload @@ -226,9 +229,8 @@ Here is how the integration could look like: ) ) -.. note:: Ensure that the event is triggered consistently and only when the event should be triggered. Avoid triggering the event multiple times for the same event unless necessary, e.g., when there is no other way to ensure that the event is triggered consistently. - -.. note:: Try placing the event after the triggering logic to ensure that the event is triggered only when the triggering logic completes successfully. This will help ensure that the event is triggered only for factual events if the triggering logic fails, the event should not be triggered. +- Ensure that the event is triggered consistently and only when the event should be triggered. Avoid triggering the event multiple times for the same event unless necessary, e.g., when there is no other way to ensure that the event is triggered consistently. +- Try placing the event after the triggering logic completes successfully to ensure that the event is triggered only when the event should be triggered. This will help ensure that the event is triggered only for factual events if the triggering logic fails, the event should not be triggered. Step 7: Test the Event ~~~~~~~~~~~~~~~~~~~~~~~ @@ -297,7 +299,9 @@ In our example, we could write a test that enrolls a user in a course and verifi event_receiver.call_args.kwargs ) -.. note:: Ensure that the test verifies that the event is triggered when the enrollment process completes successfully and that the payload contains the necessary information. +- Ensure that the test verifies that the event is triggered when the enrollment process completes successfully and that the payload contains the necessary information. +- Connect a dummy event receiver to the event to verify that the event is triggered. +- Verify that the event receiver is called with the correct payload when the event is triggered. Step 8: Consume the Event ~~~~~~~~~~~~~~~~~~~~~~~~~ From c45f5e495bf0d7dea9db45b1d563bc735037f685 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Wed, 25 Dec 2024 13:00:51 +0100 Subject: [PATCH 17/38] fix: add plural for contribution process --- docs/how-tos/create-a-new-event.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index 895597d6..13c68e09 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -311,7 +311,7 @@ Since the event is now implemented, you should consume the event to verify that Step 9: Continue the Contribution Process ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -After implementing the event, you should continue the contribution process by creating a pull request in the repository. The pull request should contain the changes you made to implement the event, including the event definition, payload, and the places where the event is triggered. +After implementing the event, you should continue the contribution process by creating a pull request in the repository. The pull requests should contain the changes you made to implement the event, including the event definition, payload, and the places where the event is triggered. For more details on how the contribution flow works, refer to the :doc:`docs.openedx.org:developers/concepts/hooks_extension_framework` documentation. From 1b5d6ab6c03839fa6cc48a1b2559dbf437199597 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Wed, 25 Dec 2024 16:33:38 +0100 Subject: [PATCH 18/38] docs: clarify identify triggering logic --- docs/how-tos/create-a-new-event.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index 13c68e09..6ec503a9 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -74,9 +74,9 @@ For the enrollment event, the event type could be ``org.openedx.learning.course. Step 3: Identify the Event Triggering Logic ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The triggering logic for the event should be identified to ensure that the event is triggered in the right places and that the event is triggered consistently. We should identify the triggering logic to ensure that maximum coverage is achieved with minimal modifications. The goal is to focus on core, critical areas where the logic we want to modify executes, ensuring the event is triggered consistently. +The triggering logic for the event should be identified to ensure that the event is triggered consistently in the right places. We should ensure that maximum coverage is achieved with minimal modifications when placing the event in the service we're modifying. The goal is to focus on core, critical areas where the logic we want to modify executes. -In our example, the triggering logic could be a place where all enrollment logic goes through. This could be the ``enroll`` method in the enrollment model in the LMS, which is called when a user enrolls in a course in all cases. +For this, choose a specific point in the service where the event should be triggered. This could be a method in a service, a view, or a model where the logic that you interested in is executed. The triggering logic should be consistent and narrow to ensure that the event is triggered only when the conditions are met. For instance, the triggering logic should be a place where all enrollment logic goes through, ensuring that the event is triggered consistently when a user enrolls in a course. This could be the ``enroll`` method in the enrollment model in the LMS, which is called when a user enrolls in a course in all cases. .. note:: When designing an event take into account the support over time of the service and triggering logic. If the service is likely to change or be deprecated, consider the implications of implementing the event in that service. From 34e9b9ddb144bc9add6de44ace39ed41253a2f95 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Wed, 25 Dec 2024 16:35:01 +0100 Subject: [PATCH 19/38] docs: clarify where to place an event --- docs/how-tos/create-a-new-event.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index 6ec503a9..78ac041d 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -195,7 +195,7 @@ The :term:`Event Definition` should be implemented in the corresponding subdomai Step 6: Send the Event ~~~~~~~~~~~~~~~~~~~~~~~ -After defining the event, you should trigger the event in the places we identified in the triggering logic. In our example, we identified that the event should be triggered when a user enrolls in a course so it should be triggered when the enrollment process completes successfully independent of the method of enrollment used. Therefore, we should trigger the event in the ``enroll`` method in the enrollment model in the LMS services. +After defining the event, you should trigger the event in the places we identified in the triggering logic. In our example, we identified that the event should be triggered when a user enrolls in a course so it should be triggered when the enrollment process completes successfully independent of the method of enrollment used. Therefore, we should trigger the event in the ``enroll`` method in the enrollment model in the LMS service when the enrollment process completes successfully, i.e., at the end of the method. Here is how the integration could look like: @@ -230,7 +230,7 @@ Here is how the integration could look like: ) - Ensure that the event is triggered consistently and only when the event should be triggered. Avoid triggering the event multiple times for the same event unless necessary, e.g., when there is no other way to ensure that the event is triggered consistently. -- Try placing the event after the triggering logic completes successfully to ensure that the event is triggered only when the event should be triggered. This will help ensure that the event is triggered only for factual events if the triggering logic fails, the event should not be triggered. +- Try placing the event after the triggering logic completes successfully to ensure that the event is triggered only when it's needed. This will help ensure that the event is triggered only for factual events if the triggering logic fails, the event should not be triggered. Step 7: Test the Event ~~~~~~~~~~~~~~~~~~~~~~~ From 28acdd6e596d0af356de4c0f69a676ce0d880e05 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Wed, 25 Dec 2024 17:34:35 +0100 Subject: [PATCH 20/38] docs: add note about inspecting event content to implement custom logic --- docs/how-tos/consume-an-event.rst | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/docs/how-tos/consume-an-event.rst b/docs/how-tos/consume-an-event.rst index 71c4c853..d876ce68 100644 --- a/docs/how-tos/consume-an-event.rst +++ b/docs/how-tos/consume-an-event.rst @@ -49,6 +49,20 @@ An :term:`Event Receiver` is simply a function that listens for a specific event - The Django dispatcher will call the ``create_notification_preference`` function when the ``COURSE_ENROLLMENT_CREATED`` event is triggered by using the ``receiver`` decorator. In this case, that would be every time a user enrolls in a course. - Consider using asynchronous tasks to handle the event processing to avoid blocking the main thread and improve performance. Also, make sure to handle exceptions and errors gracefully to avoid silent failures and improve debugging. You should also consider not creating a tight coupling between receivers and other services, if doing so is necessary consider using the event bus to broadcast the event. +- When implementing the receiver, inspect the event payload to understand the data that is being passed to the event receiver by reviewing the ``data.py`` file of the event you are consuming. For example, the ``COURSE_ENROLLMENT_CREATED`` event has the following payload: + +.. code-block:: python + + # Location openedx_events/learning/data.py + COURSE_ENROLLMENT_CREATED = OpenEdxPublicSignal( + event_type="org.openedx.learning.course.enrollment.created.v1", + data={ + "enrollment": CourseEnrollmentData, + } + ) + +- This event has a single field called ``enrollment`` which is an instance of the ``CourseEnrollmentData`` class. You can review the ``CourseEnrollmentData`` class to understand the data that is available to you and how you can use it to implement the custom logic. +- The ``metadata`` parameter contains the Open edX-specific metadata for the event, such as the event version and timestamp when the event was sent. You can use this metadata to understand more about the event and its context. Step 3: Test the Event Receiver ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From 2440a24a74e6e69a3287e3450ff5e0a5710314fa Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Wed, 25 Dec 2024 17:54:45 +0100 Subject: [PATCH 21/38] docs: use example from openedx-events-2-zapier plugin --- docs/how-tos/consume-an-event.rst | 31 +++++++++++++++++------------ docs/how-tos/create-a-new-event.rst | 2 +- 2 files changed, 19 insertions(+), 14 deletions(-) diff --git a/docs/how-tos/consume-an-event.rst b/docs/how-tos/consume-an-event.rst index d876ce68..7ffc2db9 100644 --- a/docs/how-tos/consume-an-event.rst +++ b/docs/how-tos/consume-an-event.rst @@ -21,7 +21,12 @@ Steps To consume an event within the same service, follow these steps: -Step 1: Install Open edX Events +Step 1: Understand your Use Case +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Before you start consuming an event, you should understand the use case and the event you want to consume. In this case, we want to send the enrollment data to a webhook when a user enrolls in a course. You should review the event definition and payload to understand the data that is being passed to the event receiver and how you can use it to implement the custom logic. + +Step 2: Install Open edX Events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ First, add the ``openedx-events`` plugin into your dependencies so the library's environment recognizes the event you want to consume. You can install ``openedx-events`` by running: @@ -32,7 +37,7 @@ First, add the ``openedx-events`` plugin into your dependencies so the library's This will mainly make the events available for your CI/CD pipeline and local development environment. If you are using the Open edX platform, the library should be already be installed in the environment so no need to install it. -Step 2: Create a Event Receiver and Connect it to the Event +Step 3: Create a Event Receiver and Connect it to the Event ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An :term:`Event Receiver` is simply a function that listens for a specific event and executes custom logic in response to the event being triggered. You can create an event receiver by using the Django signal receivers decorator. Here's an example of an event receiver that listens for the ``COURSE_ENROLLMENT_CREATED`` event and creates a notification preference for the user: @@ -43,11 +48,11 @@ An :term:`Event Receiver` is simply a function that listens for a specific event from django.dispatch import receiver @receiver(COURSE_ENROLLMENT_CREATED) - def create_notification_preference(signal, sender, enrollment, metadata, **kwargs): - # Custom logic to create a notification preference for the user + def send_enrollment_data_to_webhook(signal, sender, enrollment, metadata, **kwargs): + # Custom logic to send enrollment data to a webhook pass -- The Django dispatcher will call the ``create_notification_preference`` function when the ``COURSE_ENROLLMENT_CREATED`` event is triggered by using the ``receiver`` decorator. In this case, that would be every time a user enrolls in a course. +- The Django dispatcher will call the ``send_enrollment_data_to_webhook`` function when the ``COURSE_ENROLLMENT_CREATED`` event is triggered by using the ``receiver`` decorator. In this case, that would be every time a user enrolls in a course. - Consider using asynchronous tasks to handle the event processing to avoid blocking the main thread and improve performance. Also, make sure to handle exceptions and errors gracefully to avoid silent failures and improve debugging. You should also consider not creating a tight coupling between receivers and other services, if doing so is necessary consider using the event bus to broadcast the event. - When implementing the receiver, inspect the event payload to understand the data that is being passed to the event receiver by reviewing the ``data.py`` file of the event you are consuming. For example, the ``COURSE_ENROLLMENT_CREATED`` event has the following payload: @@ -64,7 +69,7 @@ An :term:`Event Receiver` is simply a function that listens for a specific event - This event has a single field called ``enrollment`` which is an instance of the ``CourseEnrollmentData`` class. You can review the ``CourseEnrollmentData`` class to understand the data that is available to you and how you can use it to implement the custom logic. - The ``metadata`` parameter contains the Open edX-specific metadata for the event, such as the event version and timestamp when the event was sent. You can use this metadata to understand more about the event and its context. -Step 3: Test the Event Receiver +Step 4: Test the Event Receiver ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Given the design of Open edX Events, you can include the events definitions in your test suite to ensure that the event receiver is working as expected. You can use the ``send_event`` method to trigger the event and test the event receiver. Here's an example of how you can test the event receiver: @@ -73,7 +78,7 @@ Given the design of Open edX Events, you can include the events definitions in y from openedx_events import send_event, COURSE_ENROLLMENT_CREATED - def test_create_notification_preference(): + def test_send_enrollment_data_to_webhook(self): # Trigger the event enrollment_data = CourseEnrollmentData( user=UserData( @@ -93,20 +98,20 @@ Given the design of Open edX Events, you can include the events definitions in y is_active=self.course_enrollment.is_active, creation_date=self.course_enrollment.created, ) + COURSE_ENROLLMENT_CREATED.send_event( enrollment=enrollment_data ) - # Assert that CourseNotificationPreference object was created with correct attributes - notification_preferences = CourseNotificationPreference.objects.all() - - self.assertEqual(notification_preferences.count(), 1) - self.assertEqual(notification_preferences[0].user, self.user) + # Assert that the request was sent to the webhook with the correct data - In the test suite, you can use the ``send_event`` method to trigger the event and pass the necessary data to the event receiver. In this case, we are passing the user, course and enrollment data to the event receiver as the triggering logic would do. -- After triggering the event, you can assert that the event receiver executed the custom logic as expected. In this case, we are checking that a ``CourseNotificationPreference`` object was created with the correct attributes. +- After triggering the event, you can assert that the event receiver executed the custom logic as expected. In this case, we are checking that the request was sent to the webhook with the correct data. + +You can review this example to understand how you can test the event receiver and ensure that the custom logic is executed when the event is triggered in the openedx-events-2-zapier plugin. This way you can ensure that the event receiver is working as expected and that the custom logic is executed when the event is triggered. If the event definition or payload changes in any way, you can catch the error in the test suite instead of in production. .. _Tutor: https://docs.tutor.edly.io/ .. _Django Signals Documentation: https://docs.djangoproject.com/en/4.2/topics/signals/ +.. _openedx-events-2-zapier: https://github.com/eduNEXT/openedx-events-2-zapier diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index 78ac041d..8c609817 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -56,7 +56,7 @@ Before contributing a new event, it is important to propose the event to the com In our example our use case proposal could be: - I want to add an event that will be triggered when a user enrolls in a course from the course about page. This event will be useful for services that need to send notifications to the user when they enroll in a course. + I want to add an event that will be triggered when a user enrolls in a course from the course about page. This event will be useful for services that need to send the enrollment data to external services for further processing. If you are confident that the event is beneficial to the community, you can proceed to the next steps and implement the event. From a0cdf57e5ce10584124123355eaebf2dfb60194e Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Wed, 25 Dec 2024 18:54:41 +0100 Subject: [PATCH 22/38] docs: match description for new use case --- docs/how-tos/consume-an-event.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/how-tos/consume-an-event.rst b/docs/how-tos/consume-an-event.rst index 7ffc2db9..2dd1c5af 100644 --- a/docs/how-tos/consume-an-event.rst +++ b/docs/how-tos/consume-an-event.rst @@ -5,7 +5,7 @@ You have two ways of consuming an Open edX event, within the same service or in .. note:: We encourage you to also consider the practices outlined in the :doc:`../decisions/0016-event-design-practices` ADR for event consumption. -Throughout this guide, we will use an example of creating an event handler that will execute when a user enrolls in a course from the course about page to better illustrate the steps involved in creating a consumer for an event. +Throughout this guide, we will implement the use case to send the enrollment data to a webhook when a user enrolls in a course to better illustrate the steps involved in creating a consumer for an event. Assumptions ----------- From dc6855e147b4c56d4e0aeeb879a8d85a272c38ca Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Wed, 25 Dec 2024 19:02:21 +0100 Subject: [PATCH 23/38] docs: add notes about reviewing the event content for the use case --- docs/how-tos/consume-an-event.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/how-tos/consume-an-event.rst b/docs/how-tos/consume-an-event.rst index 2dd1c5af..9b4fef3c 100644 --- a/docs/how-tos/consume-an-event.rst +++ b/docs/how-tos/consume-an-event.rst @@ -26,6 +26,8 @@ Step 1: Understand your Use Case Before you start consuming an event, you should understand the use case and the event you want to consume. In this case, we want to send the enrollment data to a webhook when a user enrolls in a course. You should review the event definition and payload to understand the data that is being passed to the event receiver and how you can use it to implement the custom logic. +In our example, we want to send the enrollment data to a webhook when a user enrolls in a course. We will consume the ``COURSE_ENROLLMENT_CREATED`` event, which is triggered every time a user enrolls in a course. You can review the event definition and payload to understand the data that is being passed to the event receiver and how you can use it to implement the request to the webhook. + Step 2: Install Open edX Events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From 72cca67c7d44de222be1c8f5287875a672241f7a Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Wed, 25 Dec 2024 19:18:20 +0100 Subject: [PATCH 24/38] docs: add note about django plugins --- docs/how-tos/consume-an-event.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/how-tos/consume-an-event.rst b/docs/how-tos/consume-an-event.rst index 9b4fef3c..3c013206 100644 --- a/docs/how-tos/consume-an-event.rst +++ b/docs/how-tos/consume-an-event.rst @@ -71,6 +71,8 @@ An :term:`Event Receiver` is simply a function that listens for a specific event - This event has a single field called ``enrollment`` which is an instance of the ``CourseEnrollmentData`` class. You can review the ``CourseEnrollmentData`` class to understand the data that is available to you and how you can use it to implement the custom logic. - The ``metadata`` parameter contains the Open edX-specific metadata for the event, such as the event version and timestamp when the event was sent. You can use this metadata to understand more about the event and its context. +These event receivers are implemented in `Open edX Django plugins`_ and are registered in the ``signals.py`` file of the plugin. You can review the ``signals.py`` file of the openedx-events-2-zapier_ plugin to understand how the event receivers are implemented and connected to the events. + Step 4: Test the Event Receiver ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -117,3 +119,4 @@ This way you can ensure that the event receiver is working as expected and that .. _Tutor: https://docs.tutor.edly.io/ .. _Django Signals Documentation: https://docs.djangoproject.com/en/4.2/topics/signals/ .. _openedx-events-2-zapier: https://github.com/eduNEXT/openedx-events-2-zapier +.. _Open edX Django plugins: https://docs.openedx.org/en/latest/developers/concepts/platform_overview.html#new-plugin From c5ca640c7a91e1dd996c3ff76620f387a7ec3e22 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Wed, 25 Dec 2024 19:27:49 +0100 Subject: [PATCH 25/38] docs: add note about OEP-49 handler patterns --- docs/how-tos/consume-an-event.rst | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/how-tos/consume-an-event.rst b/docs/how-tos/consume-an-event.rst index 3c013206..0653d7cd 100644 --- a/docs/how-tos/consume-an-event.rst +++ b/docs/how-tos/consume-an-event.rst @@ -71,7 +71,9 @@ An :term:`Event Receiver` is simply a function that listens for a specific event - This event has a single field called ``enrollment`` which is an instance of the ``CourseEnrollmentData`` class. You can review the ``CourseEnrollmentData`` class to understand the data that is available to you and how you can use it to implement the custom logic. - The ``metadata`` parameter contains the Open edX-specific metadata for the event, such as the event version and timestamp when the event was sent. You can use this metadata to understand more about the event and its context. -These event receivers are implemented in `Open edX Django plugins`_ and are registered in the ``signals.py`` file of the plugin. You can review the ``signals.py`` file of the openedx-events-2-zapier_ plugin to understand how the event receivers are implemented and connected to the events. +These event receivers are usually implemented independently of the service in an `Open edX Django plugins`_ and are registered in the ``handlers.py`` (according to `OEP-49`_) file of the plugin. You can review the ``handlers.py`` file of the openedx-events-2-zapier_ plugin to understand how the event receivers are implemented and connected to the events. + +.. TODO: change receivers.py in openedx-events-2-zapier to handlers.py Step 4: Test the Event Receiver ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -120,3 +122,4 @@ This way you can ensure that the event receiver is working as expected and that .. _Django Signals Documentation: https://docs.djangoproject.com/en/4.2/topics/signals/ .. _openedx-events-2-zapier: https://github.com/eduNEXT/openedx-events-2-zapier .. _Open edX Django plugins: https://docs.openedx.org/en/latest/developers/concepts/platform_overview.html#new-plugin +.. _OEP-49: https://docs.openedx.org/projects/openedx-proposals/en/latest/best-practices/oep-0049-django-app-patterns.html#signals From 5e0c04f9792fafece81cbbdcf8b3f9fb65dc22a1 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Wed, 25 Dec 2024 19:44:00 +0100 Subject: [PATCH 26/38] docs: add note about identify event to consume --- docs/how-tos/consume-an-event.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/how-tos/consume-an-event.rst b/docs/how-tos/consume-an-event.rst index 0653d7cd..eed7f98e 100644 --- a/docs/how-tos/consume-an-event.rst +++ b/docs/how-tos/consume-an-event.rst @@ -21,10 +21,10 @@ Steps To consume an event within the same service, follow these steps: -Step 1: Understand your Use Case -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Step 1: Understand your Use Case and Identify the Event to Consume +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Before you start consuming an event, you should understand the use case and the event you want to consume. In this case, we want to send the enrollment data to a webhook when a user enrolls in a course. You should review the event definition and payload to understand the data that is being passed to the event receiver and how you can use it to implement the custom logic. +Before you start consuming an event, you should understand the use case and the event you want to consume, for this review the `list of events`_ available in the Open edX platform. In this case, we want to send the enrollment data to a webhook when a user enrolls in a course. You should review the event definition and payload to understand the data that is being passed to the event receiver and how you can use it to implement the custom logic. In our example, we want to send the enrollment data to a webhook when a user enrolls in a course. We will consume the ``COURSE_ENROLLMENT_CREATED`` event, which is triggered every time a user enrolls in a course. You can review the event definition and payload to understand the data that is being passed to the event receiver and how you can use it to implement the request to the webhook. @@ -123,3 +123,4 @@ This way you can ensure that the event receiver is working as expected and that .. _openedx-events-2-zapier: https://github.com/eduNEXT/openedx-events-2-zapier .. _Open edX Django plugins: https://docs.openedx.org/en/latest/developers/concepts/platform_overview.html#new-plugin .. _OEP-49: https://docs.openedx.org/projects/openedx-proposals/en/latest/best-practices/oep-0049-django-app-patterns.html#signals +.. _list of events: https://docs.openedx.org/projects/openedx-events/en/latest/reference/events.html From 4553a7193fcdf00d158b25ba56398c12d503e098 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Thu, 26 Dec 2024 12:06:36 +0100 Subject: [PATCH 27/38] refactor: fix typing error --- docs/how-tos/create-a-new-event.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index 8c609817..6f00c042 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -306,7 +306,7 @@ In our example, we could write a test that enrolls a user in a course and verifi Step 8: Consume the Event ~~~~~~~~~~~~~~~~~~~~~~~~~ -Since the event is now implemented, you should consume the event to verify that it is triggered and that the payload contains the necessary information. You can consume the event in a test environment using a Django Signal Receiver. This will help you verify that the event is triggered and that the payload contains the necessary information. You can use follow the steps in :doc:`../how-tos/consume-an-event` to consume the event in a test environment with a Django Signal Receiver. Or you could also use the Open edX Event Bus to consume the event in a test environment. For more information on how to use the Open edX Event Bus, refer to the :doc:`../how-tos/use-the-event-bus-to-broadcast-and-consume-events`. +Since the event is now implemented, you should consume the event to verify that it is triggered and that the payload contains the necessary information. You can consume the event in a test environment using a Django Signal Receiver. This will help you verify that the event is triggered and that the payload contains the necessary information. You can follow the steps in :doc:`../how-tos/consume-an-event` to consume the event in a test environment with a Django Signal Receiver. Or you could also use the Open edX Event Bus to consume the event in a test environment. For more information on how to use the Open edX Event Bus, refer to the :doc:`../how-tos/use-the-event-bus-to-broadcast-and-consume-events`. Step 9: Continue the Contribution Process ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From b4d7326d9d3257b0612a073a51ecb96f4f8df470 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Thu, 26 Dec 2024 12:15:36 +0100 Subject: [PATCH 28/38] docs: add notes about implementing receivers --- docs/how-tos/consume-an-event.rst | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/how-tos/consume-an-event.rst b/docs/how-tos/consume-an-event.rst index eed7f98e..1c974012 100644 --- a/docs/how-tos/consume-an-event.rst +++ b/docs/how-tos/consume-an-event.rst @@ -75,6 +75,13 @@ These event receivers are usually implemented independently of the service in an .. TODO: change receivers.py in openedx-events-2-zapier to handlers.py +Consider the following when implementing the event receiver: + +- Limit each receiver to a single responsibility to make the code easier to maintain and test. +- Keep the receiver logic simple and focused on the specific task it needs to perform. +- Consider the performance implications of the receiver and avoid adding unnecessary complexity or overhead, considering that receivers will be executed each time the event is triggered. Consider using asynchronous tasks to handle the event processing to avoid blocking the main thread and improve performance. +- Implement error handling and logging in the pipeline step to handle exceptions and provide useful information for debugging, considering both development and production environments. + Step 4: Test the Event Receiver ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From ccb2bf09cc87880f01d1477062ae638d8584b816 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Thu, 26 Dec 2024 12:23:02 +0100 Subject: [PATCH 29/38] docs: add suggestion about avoiding suffixes in data field names --- docs/how-tos/create-a-new-event.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index 6f00c042..f892731d 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -187,6 +187,7 @@ The :term:`Event Definition` should be implemented in the corresponding subdomai - The ``event_name`` should be a constant that is used to identify the event in the code. - The ``event_description`` should describe what the event is about and why it is triggered. - The ``event_data`` should be the payload class that is used to define the data that is included in the event. +- The ``data`` dictionary should contain the payload class that is used to define the data that is included in the event. This will help consumers understand the event and react to it. Try using a descriptive name for the data field, but keep consistency with the payload class name. Avoid using suffixes like ``_data`` or ``_payload`` in the data field name. - The event should be an instance of the ``OpenEdxPublicSignal`` class to ensure that the event is consistent with the Open edX event framework. - Receivers should be able to access the event payload in their receivers to react to the event. From 05fa76da226f0bf341e4ca577cc04707e0681e6c Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Tue, 7 Jan 2025 16:57:03 +0100 Subject: [PATCH 30/38] refactor: address PR review --- docs/reference/architecture-subdomains.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reference/architecture-subdomains.rst b/docs/reference/architecture-subdomains.rst index c65eb991..dff2befe 100644 --- a/docs/reference/architecture-subdomains.rst +++ b/docs/reference/architecture-subdomains.rst @@ -10,7 +10,7 @@ Currently, these are the `architecture subdomains`_ used by the Open edX Events +-------------------+----------------------------------------------------------------------------------------------------+ | Learning | Allows learners to consume content and perform actions in a learning activity on the platform. | +-------------------+----------------------------------------------------------------------------------------------------+ -| Analytics | Provides insights into learner behavior and course performance. | +| Analytics | Provides visibility into learner behavior and course performance. | +-------------------+----------------------------------------------------------------------------------------------------+ | Enterprise | Provides tools for organizations to manage their learners and courses. | +-------------------+----------------------------------------------------------------------------------------------------+ From 594908aec0a6bfd4eb583a98419915a3164d5b1e Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Wed, 8 Jan 2025 15:22:58 +0100 Subject: [PATCH 31/38] docs: drop unnecessary assumption --- docs/how-tos/consume-an-event.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/how-tos/consume-an-event.rst b/docs/how-tos/consume-an-event.rst index 1c974012..26d9d0de 100644 --- a/docs/how-tos/consume-an-event.rst +++ b/docs/how-tos/consume-an-event.rst @@ -12,7 +12,6 @@ Assumptions - You have a development environment set up using `Tutor`_. - You have a basic understanding of Python and Django. -- You have created a new Open edX event. If not, you can follow the :doc:`../how-tos/create-a-new-event` guide to create a new event. - You have a basic understanding of Django signals. If not, you can review the `Django Signals Documentation`_. - You are familiar with the terminology used in the project, such as the terms :term:`Event Type` or :term:`Event Receiver`. If not, you can review the :doc:`../reference/glossary` documentation. From bedf650a782cff7080bbdc3c226c770dcda87a88 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Wed, 8 Jan 2025 15:23:12 +0100 Subject: [PATCH 32/38] docs: add reference to openedx-events-2-zapier --- docs/how-tos/consume-an-event.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/how-tos/consume-an-event.rst b/docs/how-tos/consume-an-event.rst index 26d9d0de..3dba2e60 100644 --- a/docs/how-tos/consume-an-event.rst +++ b/docs/how-tos/consume-an-event.rst @@ -70,7 +70,7 @@ An :term:`Event Receiver` is simply a function that listens for a specific event - This event has a single field called ``enrollment`` which is an instance of the ``CourseEnrollmentData`` class. You can review the ``CourseEnrollmentData`` class to understand the data that is available to you and how you can use it to implement the custom logic. - The ``metadata`` parameter contains the Open edX-specific metadata for the event, such as the event version and timestamp when the event was sent. You can use this metadata to understand more about the event and its context. -These event receivers are usually implemented independently of the service in an `Open edX Django plugins`_ and are registered in the ``handlers.py`` (according to `OEP-49`_) file of the plugin. You can review the ``handlers.py`` file of the openedx-events-2-zapier_ plugin to understand how the event receivers are implemented and connected to the events. +These event receivers are usually implemented independently of the service in an `Open edX Django plugins`_ and are registered in the ``handlers.py`` (according to `OEP-49`_) file of the plugin. You can review the ``handlers.py`` file of the `openedx-events-2-zapier`_ plugin to understand how the event receivers are implemented and connected to the events. .. TODO: change receivers.py in openedx-events-2-zapier to handlers.py @@ -120,7 +120,7 @@ Given the design of Open edX Events, you can include the events definitions in y - In the test suite, you can use the ``send_event`` method to trigger the event and pass the necessary data to the event receiver. In this case, we are passing the user, course and enrollment data to the event receiver as the triggering logic would do. - After triggering the event, you can assert that the event receiver executed the custom logic as expected. In this case, we are checking that the request was sent to the webhook with the correct data. -You can review this example to understand how you can test the event receiver and ensure that the custom logic is executed when the event is triggered in the openedx-events-2-zapier plugin. +You can review this example to understand how you can test the event receiver and ensure that the custom logic is executed when the event is triggered in the `openedx-events-2-zapier`_ plugin. This way you can ensure that the event receiver is working as expected and that the custom logic is executed when the event is triggered. If the event definition or payload changes in any way, you can catch the error in the test suite instead of in production. From 131a5e30207d19f614a0e0422a262cfd11c1609d Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Fri, 10 Jan 2025 12:39:12 +0100 Subject: [PATCH 33/38] docs: apply suggestions from code review Co-authored-by: Sarina Canelake --- docs/how-tos/consume-an-event.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/how-tos/consume-an-event.rst b/docs/how-tos/consume-an-event.rst index 3dba2e60..a2ea7642 100644 --- a/docs/how-tos/consume-an-event.rst +++ b/docs/how-tos/consume-an-event.rst @@ -54,7 +54,7 @@ An :term:`Event Receiver` is simply a function that listens for a specific event pass - The Django dispatcher will call the ``send_enrollment_data_to_webhook`` function when the ``COURSE_ENROLLMENT_CREATED`` event is triggered by using the ``receiver`` decorator. In this case, that would be every time a user enrolls in a course. -- Consider using asynchronous tasks to handle the event processing to avoid blocking the main thread and improve performance. Also, make sure to handle exceptions and errors gracefully to avoid silent failures and improve debugging. You should also consider not creating a tight coupling between receivers and other services, if doing so is necessary consider using the event bus to broadcast the event. +- Consider using asynchronous tasks to handle the event processing to avoid blocking the main thread and improve performance. Also, make sure to handle exceptions and errors gracefully to avoid silent failures and improve debugging. It is recommended to not create a tight coupling between receivers and other services. If doing so is necessary consider using the event bus to broadcast the event. - When implementing the receiver, inspect the event payload to understand the data that is being passed to the event receiver by reviewing the ``data.py`` file of the event you are consuming. For example, the ``COURSE_ENROLLMENT_CREATED`` event has the following payload: .. code-block:: python From 00a8d3fa128647cd446db1df123981c99f4b98fd Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Fri, 10 Jan 2025 12:51:47 +0100 Subject: [PATCH 34/38] refactor: address PR reviews --- docs/how-tos/consume-an-event.rst | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/how-tos/consume-an-event.rst b/docs/how-tos/consume-an-event.rst index a2ea7642..95ec15eb 100644 --- a/docs/how-tos/consume-an-event.rst +++ b/docs/how-tos/consume-an-event.rst @@ -72,8 +72,6 @@ An :term:`Event Receiver` is simply a function that listens for a specific event These event receivers are usually implemented independently of the service in an `Open edX Django plugins`_ and are registered in the ``handlers.py`` (according to `OEP-49`_) file of the plugin. You can review the ``handlers.py`` file of the `openedx-events-2-zapier`_ plugin to understand how the event receivers are implemented and connected to the events. -.. TODO: change receivers.py in openedx-events-2-zapier to handlers.py - Consider the following when implementing the event receiver: - Limit each receiver to a single responsibility to make the code easier to maintain and test. From 069844cd92a8f950afe6decd4f6acbab2a5e02f6 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Wed, 15 Jan 2025 11:55:07 +0100 Subject: [PATCH 35/38] refactor: use infinitive instead of continuous form for titles --- ...-to-an-event.rst => add-event-bus-support-to-an-event.rst} | 4 ++-- docs/how-tos/index.rst | 4 ++-- .../{using-the-event-bus.rst => use-the-event-bus.rst} | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) rename docs/how-tos/{adding-event-bus-support-to-an-event.rst => add-event-bus-support-to-an-event.rst} (99%) rename docs/how-tos/{using-the-event-bus.rst => use-the-event-bus.rst} (99%) diff --git a/docs/how-tos/adding-event-bus-support-to-an-event.rst b/docs/how-tos/add-event-bus-support-to-an-event.rst similarity index 99% rename from docs/how-tos/adding-event-bus-support-to-an-event.rst rename to docs/how-tos/add-event-bus-support-to-an-event.rst index 9138c699..1d2f24a3 100644 --- a/docs/how-tos/adding-event-bus-support-to-an-event.rst +++ b/docs/how-tos/add-event-bus-support-to-an-event.rst @@ -1,5 +1,5 @@ -Adding Event Bus Support to an Open edX Event -============================================= +Add Event Bus Support to an Open edX Event +========================================== Before sending an event across services, you need to ensure that the event is compatible with the Open edX Event Bus. This involves ensuring that the event, with its corresponding payload, can be emitted by a service through the event bus and that it can be consumed by other services. This guide will walk you through the process of adding event bus support to an Open edX event. diff --git a/docs/how-tos/index.rst b/docs/how-tos/index.rst index 325d0bd6..73f930c5 100644 --- a/docs/how-tos/index.rst +++ b/docs/how-tos/index.rst @@ -7,7 +7,7 @@ How-tos create-a-new-event consume-an-event - adding-event-bus-support-to-an-event + add-event-bus-support-to-an-event use-the-event-bus-to-broadcast-and-consume-events - using-the-event-bus + use-the-event-bus add-new-event-bus-concrete-implementation diff --git a/docs/how-tos/using-the-event-bus.rst b/docs/how-tos/use-the-event-bus.rst similarity index 99% rename from docs/how-tos/using-the-event-bus.rst rename to docs/how-tos/use-the-event-bus.rst index 69d79c15..49ca5e8b 100644 --- a/docs/how-tos/using-the-event-bus.rst +++ b/docs/how-tos/use-the-event-bus.rst @@ -1,5 +1,5 @@ -Using the Open edX Event Bus -============================ +Use the Open edX Event Bus +========================== .. note:: Be sure to check out how to make your Open edX Event event bus compatible in the :doc:`../how-tos/adding-event-bus-support-to-an-event` guide. From 1997569c057ac65e9f9aa2aaadfcf3700456cf6f Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Wed, 15 Jan 2025 12:00:02 +0100 Subject: [PATCH 36/38] fix: address doc rendering errors --- docs/how-tos/add-event-bus-support-to-an-event.rst | 4 ++-- docs/how-tos/use-the-event-bus.rst | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/how-tos/add-event-bus-support-to-an-event.rst b/docs/how-tos/add-event-bus-support-to-an-event.rst index 1d2f24a3..42c8b952 100644 --- a/docs/how-tos/add-event-bus-support-to-an-event.rst +++ b/docs/how-tos/add-event-bus-support-to-an-event.rst @@ -6,7 +6,7 @@ Before sending an event across services, you need to ensure that the event is co For more details on how the :term:`Event Payload` is structured refer to the :doc:`../decisions/0003-events-payload` decision record. .. note:: - This guide assumes that you have already created an Open edX event. If you haven't, refer to the :doc:`../how-tos/creating-new-events` how-to guide. + This guide assumes that you have already created an Open edX event. If you haven't, refer to the :doc:`../how-tos/create-a-new-event` how-to guide. Step 1: Does my Event Need Event Bus Support? ---------------------------------------------- @@ -124,7 +124,7 @@ Run ``python manage.py generate_avro_schemas --help`` to see the available optio Step 5: Send the Event Across Services with the Event Bus --------------------------------------------------------- -To validate that you can consume the event emitted by a service through the event bus, you can send the event across services. Here is an example of how you can send the event across services using the Redis event bus implementation following the `setup instructions in a Tutor environment`_. We recommend also following :doc:`../how-tos/using-the-event-bus` to understand how to use the event bus in your environment. +To validate that you can consume the event emitted by a service through the event bus, you can send the event across services. Here is an example of how you can send the event across services using the Redis event bus implementation following the `setup instructions in a Tutor environment`_. We recommend also following :doc:`../how-tos/use-the-event-bus` to understand how to use the event bus in your environment. .. note:: If you implemented a custom serializer for a type in the :term:`Event Payload`, the custom serializer support must be included in both the producer and consumer sides before it can be used. diff --git a/docs/how-tos/use-the-event-bus.rst b/docs/how-tos/use-the-event-bus.rst index 49ca5e8b..3fbad3e9 100644 --- a/docs/how-tos/use-the-event-bus.rst +++ b/docs/how-tos/use-the-event-bus.rst @@ -1,7 +1,7 @@ Use the Open edX Event Bus ========================== -.. note:: Be sure to check out how to make your Open edX Event event bus compatible in the :doc:`../how-tos/adding-event-bus-support-to-an-event` guide. +.. note:: Be sure to check out how to make your Open edX Event event bus compatible in the :doc:`../how-tos/add-event-bus-support-to-an-event` guide. After creating a new Open edX Event, you might need to send it across services instead of just within the same process. For this kind of use-cases, you might want to use the Open edX Event Bus. Here :doc:`../concepts/event-bus`, you can find useful information to start getting familiar with the Open edX Event Bus. From 4bf071af3bea268dec835b88a10bc88ab84c9044 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Wed, 15 Jan 2025 12:06:28 +0100 Subject: [PATCH 37/38] docs: add reference to adding event bus support when creating a new event --- docs/how-tos/create-a-new-event.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index f892731d..7889d33c 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -163,6 +163,8 @@ In our example, the event definition and payload for the enrollment event could - Each field in the payload should be documented with a description of what the field represents and the data type it should contain. This will help consumers understand the payload and react to the event. You should be able to justify why each field is included in the payload and how it relates to the event. - Use defaults for optional fields in the payload to ensure its consistency in all cases. +.. note:: When defining the payload, enforce event bus compatibility by ensuring that the data types used in the payload align with the event bus schema format. This will help ensure that the event can be sent by the producer and be then re-emitted by the same instance of `OpenEdxPublicSignal`_ on the consumer side, guaranteeing that the data sent and received is the identical. For more information about adding event bus support to an event, refer to :doc:`../how-tos/add-event-bus-support-to-an-event`. + Event Definition **************** @@ -191,8 +193,6 @@ The :term:`Event Definition` should be implemented in the corresponding subdomai - The event should be an instance of the ``OpenEdxPublicSignal`` class to ensure that the event is consistent with the Open edX event framework. - Receivers should be able to access the event payload in their receivers to react to the event. -.. TODO: add reference to how to add event bus support to the event's payload - Step 6: Send the Event ~~~~~~~~~~~~~~~~~~~~~~~ @@ -321,3 +321,4 @@ For more details on how the contribution flow works, refer to the :doc:`docs.ope .. _attrs: https://www.attrs.org/en/stable/ .. _Tutor: https://docs.tutor.edly.io/ .. _Django Signals Documentation: https://docs.djangoproject.com/en/4.2/topics/signals/ +.. _OpenEdxPublicSignal: https://github.com/openedx/openedx-events/blob/main/openedx_events/tooling.py#L37 From cd1f81828b37d5ad4cb5de6522e8ea6e25107a10 Mon Sep 17 00:00:00 2001 From: Maria Grimaldi Date: Wed, 15 Jan 2025 12:10:43 +0100 Subject: [PATCH 38/38] docs: reference event bus concepts document --- docs/how-tos/create-a-new-event.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/how-tos/create-a-new-event.rst b/docs/how-tos/create-a-new-event.rst index 7889d33c..63a33951 100644 --- a/docs/how-tos/create-a-new-event.rst +++ b/docs/how-tos/create-a-new-event.rst @@ -163,7 +163,7 @@ In our example, the event definition and payload for the enrollment event could - Each field in the payload should be documented with a description of what the field represents and the data type it should contain. This will help consumers understand the payload and react to the event. You should be able to justify why each field is included in the payload and how it relates to the event. - Use defaults for optional fields in the payload to ensure its consistency in all cases. -.. note:: When defining the payload, enforce event bus compatibility by ensuring that the data types used in the payload align with the event bus schema format. This will help ensure that the event can be sent by the producer and be then re-emitted by the same instance of `OpenEdxPublicSignal`_ on the consumer side, guaranteeing that the data sent and received is the identical. For more information about adding event bus support to an event, refer to :doc:`../how-tos/add-event-bus-support-to-an-event`. +.. note:: When defining the payload, enforce :doc:`../concepts/event-bus` compatibility by ensuring that the data types used in the payload align with the event bus schema format. This will help ensure that the event can be sent by the producer and be then re-emitted by the same instance of `OpenEdxPublicSignal`_ on the consumer side, guaranteeing that the data sent and received is the identical. For more information about adding event bus support to an event, refer to :doc:`../how-tos/add-event-bus-support-to-an-event`. Event Definition ****************