feat(post): reana 0.9.4 release announcement (#54)

content/posts/2024/reana-0.9.4.md

@@ -0,0 +1,173 @@
+---
+title: "REANA 0.9.4 is released"
+date: 2024-12-05T06:00:00+01:00
+---
+
+REANA 0.9.4 is now available! This is a minor update that upgrades Jupyter notebooks to the latest available version, adds support for Compute4PUNCH, fixes some integration issues with HTCondor, improves the security of the platform, and more!
+
+<!--more-->
+
+## What's new for the users?
+
+### Upgraded and improved Jupyter notebooks
+
+REANA 0.9.4 now supports Jupyter Notebook 7, which comes with many improvements and bug fixes. Among many changes, Jupyter notebooks now provide a visual debugger, theming and dark mode, and a compact view on mobile devices.
+
+Furthermore, REANA secrets are now exposed in Jupyter notebooks, either as environment variables or as files. This is particularly useful if your workflow accesses restricted resources, as you are now able to authenticate easily also from Jupyter notebooks.
+
+{{< screenshot-browser-mockup src="/images/reana-0.9.4-jupyter.png" alt="Jupyter notebook showing access to secrets" >}}
+
+### HTCondor enhancements and fixes
+
+REANA supports running workflows on HTCondor using unpacked Singularity images available on CVMFS, but due to a bug up until now job commands were not executed inside the provided images. Additionally, Snakemake rules with multiline shell commands were not parsed correctly when executed on HTCondor, resulting in workflow failures. Both of these issues are now fixed in REANA 0.9.4.
+
+```yaml {hl_lines=[9,13,14,15,16]}
+rule helloworld:
+    input:
+        "input.txt"
+    output:
+        "output.txt"
+    resources:
+        kerberos=True,
+        compute_backend="htcondorcern",
+        unpacked_img=True
+    container:
+        "/cvmfs/unpacked.cern.ch/registry.hub.docker.com/library/python:3.10"
+    shell:
+        """
+        echo $SINGULARITY_NAME
+        python --version
+        """
+```
+
+### Run workflows on Compute4PUNCH
+
+REANA is now integrated with the PUNCH4NFDI infrastructure, and it is now possible to run workflows and jobs on the Compute4PUNCH HTCondor cluster. You can check whether the REANA instance you are connected to supports Compute4PUNCH with `reana-client`:
+
+```console
+$ reana-client info
+List of supported compute backends: kubernetes, compute4punch
+...
+```
+
+If you see `comput4punch` in the list of supported compute backends, then you can use the Compute4PUNCH HTCondor cluster to run some or all the steps of your workflow:
+
+```yaml {hl_lines=[4]}
+steps:
+  - name: reana_demo_helloworld_compute4punch
+    environment: "python:2.7-slim"
+    compute_backend: compute4punch
+    commands:
+      - python helloworld.py
+```
+
+## What's new for the administrators?
+
+### Platform security improvements
+
+REANA 0.9.4 greatly improves the security of the platform.
+
+First of all, it is now possible to migrate the secret key used by REANA for a number of security-related mechanisms, including database column encryption, with the new `reana-admin migrate-secret-key` command. The secret key is also now correctly propagated to all the Invenio dependencies.
+
+Additionally, REANA can now be configured to connect to password-protected instances of Redis and RabbitMQ. Passwords for the Redis and RabbitMQ instances provided by REANA's Helm chart can be customised with the `secrets.cache.password` and `secrets.message_broker.password` Helm values.
+
+Furthermore, the security context of workflow and job pods is now customised to set `allowPrivilegeEscalation` to false, to avoid jobs being able to gain additional privileges than what they were given originally.
+
+Finally, many of the security mechanisms used by REANA can now be easily configured with environment variables:
+
+- `APP_DEFAULT_SECURE_HEADERS` can be used to configure [Flask-Talisman's settings](https://github.com/GoogleCloudPlatform/flask-talisman?tab=readme-ov-file#options).
+- `REANA_FORCE_HTTPS` can be used to disable the automatic redirection of requests to HTTPS.
+- `PROXYFIX_CONFIG` can be used to correctly configure [Werkzeug's ProxyFix](https://werkzeug.palletsprojects.com/en/stable/middleware/proxy_fix/) when REANA is served behind multiple proxy servers.
+
+### Integration with Compute4PUNCH
+
+REANA 0.9.4 adds support to execute jobs on the Compute4PUNCH HTCondor cluster.
+If you want to offer this feature to your users, you have to add `compute4punch` to the list of supported compute backends in your Helm values:
+
+```yaml {hl_lines=3}
+compute_backends:
+  - kubernetes
+  - compute4punch
+```
+
+You also need to modify `components.reana_job_controller.image` to use a Docker image that includes support for Compute4PUNCH. You can either use an already published Docker image like `docker.io/reanahub/reana-job-controller-compute4punch:0.9.4` or you can even build it yourself with `reana-dev`:
+
+```console
+$ reana-dev docker-build -c reana-job-controller -b COMPUTE_BACKENDS=kubernetes,compute4punch
+```
+
+### ... and more!
+
+This new REANA release includes also more fixes and enhancements to improve the deployment and stability of the platform:
+
+- `Ingress`es needed to access interactive sessions will now set the correct hostname, solving some reported network exposure issues in some Kubernetes configurations.
+- `NetworkPolicy`s were amended to allow the periodic cronjob that closes interactive sessions to connect to the Jupyter notebook themselves to check their inactivity periods.
+- The `set_workflow_status` endpoint has been patched to make sure that submitted workflows are not able to skip the queue and be executed immediately.
+
+### How to upgrade existing REANA 0.9.3 clusters
+
+If you are a REANA cluster administrator and you would like to upgrade from
+REANA 0.9.3 to REANA 0.9.4, you can proceed as follows.
+
+Firstly, please note that you can optionally define some of the new [Helm
+values](https://github.com/reanahub/reana/blob/0.9.4/helm/reana/README.md),
+notably:
+
+- `secrets.cache.password` to set the password for the deployed Redis instance.
+- `secrets.message_broker.user` to set the username of the user account for the RabbitMQ instance.
+- `secrets.message_broker.password` to set the password of the user account to access the RabbitMQ instance.
+
+You can edit your Helm values file to add any desired new values:
+
+```console
+$ vim myvalues.yaml
+```
+
+This is however fully optional and you can simply reuse your current
+`myvalues.yaml` file without any changes to perform the 0.9.3 to 0.9.4 upgrade.
+
+If you want to create a new password-protected user for RabbitMQ, connect to its pod and run the following commands:
+
+```console
+$ rabbitmqctl add_user "<new-username>" "<new-password>"
+$ rabbitmqctl set_user_tags "<new-username>" administrator
+$ rabbitmqctl set_permissions -p "/" "<new-username>" ".*" ".*" ".*"
+```
+
+You should then set the user and password in the corresponding secrets `secrets.message_broker.user` and `secrets.message_broker.password` Helm values.
+
+Secondly, you can use the [Helm diff
+plugin](https://github.com/databus23/helm-diff) to inspect the forthcoming
+changes and then perform the upgrade using the standard Helm commands:
+
+```console
+$ helm repo update
+$ helm diff upgrade reana reanahub/reana --version 0.9.4 --values myvalues.yaml
+$ helm upgrade reana reanahub/reana --version 0.9.4 --values myvalues.yaml
+```
+
+After that, if you have created and set up a new user account for RabbitMQ, you should delete the original one:
+
+```
+rabbitmqctl delete_user "test"
+```
+
+Finally, you have to run the following command from the `reana-server` pod to re-build the encrypted database columns using the correct secret key. Note that `CHANGE_ME` needs to be copy-pasted as is and it should not be changed.
+
+```console
+$ invenio instance migrate-secret-key --old-key "CHANGE_ME"
+```
+
+If you have also decided to change your secret key at the same time, you have to additionally run the following command:
+
+```console
+$ flask reana-admin migrate-secret-key --old-key "<previous-secret-key>"
+```
+
+### More information
+
+Please see the detailed [REANA 0.9.4 release
+notes](https://github.com/reanahub/reana/releases/tag/0.9.4) for the complete
+list of all changes.
+
+Enjoy!