make jupyterhub idle server more easily configurable (#390)

## Overview

Add new variables to easily configure idle jupyter user instances.

## Changes

**Non-breaking changes**
- Jupyterhub configurable idle server culling.
  - Add optional variables `JUPYTER_IDLE_SERVER_CULL_TIMEOUT`, `JUPYTER_IDLE_KERNEL_CULL_TIMEOUT` and
    `JUPYTER_IDLE_KERNEL_CULL_INTERVAL` that allows fined-grained configuration of user-kernel and server-wide
    docker image culling when their activity status reached a certain idle timeout threshold.
  - Enable idle kernel culling by default with a timeout of 1 day, and user server culling with timeout of 3 days.
  - Avoids the need for custom `JUPYTERHUB_CONFIG_OVERRIDE` specifically for idle server culling.
    If similar argument parameters should be defined using an older `JUPYTERHUB_CONFIG_OVERRIDE` definition,
    the new configuration strategy can be skipped by setting `JUPYTER_IDLE_KERNEL_CULL_TIMEOUT=0`.

**Breaking changes**
- n/a

## Related Issue / Discussion

- Closes #389 (replaces)
- Closes Ouranosinc/jupyterhub#21 (not required anymore)

Author: fmigneault

.bumpversion.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 1.34.0
+current_version = 1.35.0
 commit = True
 tag = False
 tag_name = {new_version}
@@ -30,11 +30,11 @@ search = {current_version}
 replace = {new_version}
 
 [bumpversion:file:RELEASE.txt]
-search = {current_version} 2023-10-10T15:33:10Z
+search = {current_version} 2023-10-16T14:37:32Z
 replace = {new_version} {utcnow:%Y-%m-%dT%H:%M:%SZ}
 
 [bumpversion:part:releaseTime]
-values = 2023-10-10T15:33:10Z
+values = 2023-10-16T14:37:32Z
 
 [bumpversion:file(version):birdhouse/config/canarie-api/docker_configuration.py.template]
 search = 'version': '{current_version}'

CHANGES.md

@@ -17,6 +17,19 @@
 
 [//]: # (list changes here, using '-' for each new entry, remove this when items are added)
 
+[1.35.0](https://github.com/bird-house/birdhouse-deploy/tree/1.35.0) (2023-10-16)
+------------------------------------------------------------------------------------------------------------------
+
+## Changes
+- Jupyterhub configurable idle server culling.
+  - Add optional variables `JUPYTER_IDLE_SERVER_CULL_TIMEOUT`, `JUPYTER_IDLE_KERNEL_CULL_TIMEOUT` and
+    `JUPYTER_IDLE_KERNEL_CULL_INTERVAL` that allows fined-grained configuration of user-kernel and server-wide
+    docker image culling when their activity status reached a certain idle timeout threshold.
+  - Enable idle kernel culling by default with a timeout of 1 day, and user server culling with timeout of 3 days.
+  - Avoids the need for custom `JUPYTERHUB_CONFIG_OVERRIDE` specifically for idle server culling.
+    If similar argument parameters should be defined using an older `JUPYTERHUB_CONFIG_OVERRIDE` definition,
+    the new configuration strategy can be skipped by setting `JUPYTER_IDLE_KERNEL_CULL_TIMEOUT=0`.
+
 [1.34.0](https://github.com/bird-house/birdhouse-deploy/tree/1.34.0) (2023-10-10)
 ------------------------------------------------------------------------------------------------------------------
 

Makefile

@@ -1,7 +1,7 @@
 # Generic variables
 override SHELL       := bash
 override APP_NAME    := birdhouse-deploy
-override APP_VERSION := 1.34.0
+override APP_VERSION := 1.35.0
 
 # utility to remove comments after value of an option variable
 override clean_opt = $(shell echo "$(1)" | $(_SED) -r -e "s/[ '$'\t'']+$$//g")

README.rst

@@ -14,13 +14,13 @@ for a full-fledged production platform.
     * - releases
       - | |latest-version| |commits-since|
 
-.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.34.0.svg
+.. |commits-since| image:: https://img.shields.io/github/commits-since/bird-house/birdhouse-deploy/1.35.0.svg
     :alt: Commits since latest release
-    :target: https://github.com/bird-house/birdhouse-deploy/compare/1.34.0...master
+    :target: https://github.com/bird-house/birdhouse-deploy/compare/1.35.0...master
 
-.. |latest-version| image:: https://img.shields.io/badge/tag-1.34.0-blue.svg?style=flat
+.. |latest-version| image:: https://img.shields.io/badge/tag-1.35.0-blue.svg?style=flat
     :alt: Latest Tag
-    :target: https://github.com/bird-house/birdhouse-deploy/tree/1.34.0
+    :target: https://github.com/bird-house/birdhouse-deploy/tree/1.35.0
 
 .. |readthedocs| image:: https://readthedocs.org/projects/birdhouse-deploy/badge/?version=latest
     :alt: ReadTheDocs Build Status (latest version)

RELEASE.txt

@@ -1 +1 @@
-1.34.0 2023-10-10T15:33:10Z
+1.35.0 2023-10-16T14:37:32Z

birdhouse/config/canarie-api/docker_configuration.py.template

@@ -109,8 +109,8 @@ SERVICES = {
             # NOTE:
             #   Below version and release time auto-managed by 'make VERSION=x.y.z bump'.
             #   Do NOT modify it manually. See 'Tagging policy' in 'birdhouse/README.rst'.
-            'version': '1.34.0',
-            'releaseTime': '2023-10-10T15:33:10Z',
+            'version': '1.35.0',
+            'releaseTime': '2023-10-16T14:37:32Z',
             'institution': 'Ouranos',
             'researchSubject': 'Climatology',
             'supportEmail': '${SUPPORT_EMAIL}',
@@ -142,8 +142,8 @@ PLATFORMS = {
             # NOTE:
             #   Below version and release time auto-managed by 'make VERSION=x.y.z bump'.
             #   Do NOT modify it manually. See 'Tagging policy' in 'birdhouse/README.rst'.
-            'version': '1.34.0',
-            'releaseTime': '2023-10-10T15:33:10Z',
+            'version': '1.35.0',
+            'releaseTime': '2023-10-16T14:37:32Z',
             'institution': 'Ouranos',
             'researchSubject': 'Climatology',
             'supportEmail': '${SUPPORT_EMAIL}',

birdhouse/config/jupyterhub/default.env

@@ -43,6 +43,18 @@ export JUPYTER_LOGIN_BANNER_BOTTOM_SECTION=""
 # server for the change to take effect.
 export JUPYTERHUB_README=""
 
+# Timeout (in seconds, default: 3 days) to shut down the user server when no kernels or terminals
+# are running and there is no activity. If undefined or set to zero, the feature will not be enabled.
+export JUPYTER_IDLE_SERVER_CULL_TIMEOUT=259200
+# Timeout (in seconds, default: 1 day) after which individual
+# user kernels/terminals are considered idle and ready to be culled.
+export JUPYTER_IDLE_KERNEL_CULL_TIMEOUT=86400
+# Interval (in seconds) on which to check for idle kernels exceeding the cull timeout value.
+# Enabled only if 'JUPYTER_IDLE_KERNEL_CULL_TIMEOUT' is provided and greater than zero.
+# If this value is not provided, equal to zero, or is set higher than 'JUPYTER_IDLE_KERNEL_CULL_TIMEOUT',
+# it will be automatically reduced by half of the timeout value to ensure that it can be effective.
+export JUPYTER_IDLE_KERNEL_CULL_INTERVAL=0
+
 # Allow for adding new config or override existing config in
 # config/jupyterhub/jupyterhub_config.py.template.
 export JUPYTERHUB_CONFIG_OVERRIDE=""
@@ -68,6 +80,9 @@ OPTIONAL_VARS="
   \$JUPYTERHUB_CONFIG_OVERRIDE
   \$JUPYTERHUB_DOCKER
   \$JUPYTERHUB_VERSION
+  \$JUPYTER_IDLE_SERVER_CULL_TIMEOUT
+  \$JUPYTER_IDLE_KERNEL_CULL_TIMEOUT
+  \$JUPYTER_IDLE_KERNEL_CULL_INTERVAL
 "
 
 # add any component that this component requires to run

birdhouse/config/jupyterhub/jupyterhub_config.py.template

@@ -187,4 +187,38 @@ blocked_users = {'authtest', '${CATALOG_USERNAME}', 'anonymous'}
 c.Authenticator.blacklist = blocked_users  # v0.9+
 c.Authenticator.blocked_users = blocked_users  # v1.2+
 
+
+# ------------------------------------------------------------------------------
+# Shutdown idle user server based on configured timeouts.
+# ------------------------------------------------------------------------------
+# Timeout (in seconds, default: 3 days) to shut down the user server when no kernels or terminals
+# are running and there is no activity. If undefined or set to zero, the feature will not be enabled.
+jupyter_idle_server_cull_timeout = int("${JUPYTER_IDLE_SERVER_CULL_TIMEOUT}" or 0)
+if jupyter_idle_server_cull_timeout:
+    c.Spawner.args.append('--NotebookApp.shutdown_no_activity_timeout={}'.format(jupyter_idle_server_cull_timeout))
+# Timeout (in seconds, default: 1 day) after which individual
+# user kernels/terminals are considered idle and ready to be culled.
+jupyter_idle_kernel_cull_timeout = int("${JUPYTER_IDLE_KERNEL_CULL_TIMEOUT}" or 0)
+# Interval (in seconds, default: half of timeout) on which to check for idle kernels exceeding the cull timeout value.
+jupyter_idle_kernel_cull_interval = int("${JUPYTER_IDLE_KERNEL_CULL_INTERVAL}" or 0)
+if jupyter_idle_kernel_cull_timeout:
+    if not jupyter_idle_kernel_cull_interval or jupyter_idle_kernel_cull_interval > jupyter_idle_kernel_cull_timeout:
+        jupyter_idle_kernel_cull_interval = jupyter_idle_kernel_cull_timeout / 2
+    c.Spawner.args.extend([
+        '--MappingKernelManager.cull_idle_timeout={}'.format(jupyter_idle_kernel_cull_timeout),
+        '--MappingKernelManager.cull_interval={}'.format(jupyter_idle_kernel_cull_interval),
+        '--TerminalManager.cull_inactive_timeout={}'.format(jupyter_idle_kernel_cull_timeout),
+        '--TerminalManager.cull_interval={}'.format(jupyter_idle_kernel_cull_interval),
+    ])
+# Culling kernels which have one or more connections for idle but open notebooks and/or terminals.
+# Otherwise, browser tabs, notebooks and terminals all have to be closed for culling to work.
+if jupyter_idle_server_cull_timeout or jupyter_idle_kernel_cull_timeout:
+    c.Spawner.args.extend([
+        '--MappingKernelManager.cull_connected=True',
+        '--TerminalManager.cull_connected=True',
+    ])
+
+# ------------------------------------------------------------------------------
+# Configuration overrides
+# ------------------------------------------------------------------------------
 ${JUPYTERHUB_CONFIG_OVERRIDE}    # noqa

birdhouse/env.local.example

@@ -267,17 +267,18 @@ export GEOSERVER_ADMIN_PASSWORD=geoserverpass
 # allow jupyterhub user selection of which notebook image to run
 # see https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html
 #export ENABLE_JUPYTERHUB_MULTI_NOTEBOOKS="
-#c.DockerSpawner.image_whitelist = {os.environ['JUPYTERHUB_IMAGE_SELECTION_NAMES'].split()[0]: os.environ['DOCKER_NOTEBOOK_IMAGES'].split()[0],
-#                                   os.environ['JUPYTERHUB_IMAGE_SELECTION_NAMES'].split()[1]: os.environ['DOCKER_NOTEBOOK_IMAGES'].split()[1],
-#                                   os.environ['JUPYTERHUB_IMAGE_SELECTION_NAMES'].split()[2]: os.environ['DOCKER_NOTEBOOK_IMAGES'].split()[2],
-#                                   os.environ['JUPYTERHUB_IMAGE_SELECTION_NAMES'].split()[3]: os.environ['DOCKER_NOTEBOOK_IMAGES'].split()[3],
-#                                   'jupyter/scipy-notebook': 'jupyter/scipy-notebook',
-#                                   'jupyter/r-notebook': 'jupyter/r-notebook',
-#                                   'jupyter/tensorflow-notebook': 'jupyter/tensorflow-notebook',
-#                                   'jupyter/datascience-notebook': 'jupyter/datascience-notebook',
-#                                   'jupyter/pyspark-notebook': 'jupyter/pyspark-notebook',
-#                                   'jupyter/all-spark-notebook': 'jupyter/all-spark-notebook',
-#                                   }
+#c.DockerSpawner.image_whitelist = {
+#    os.environ['JUPYTERHUB_IMAGE_SELECTION_NAMES'].split()[0]: os.environ['DOCKER_NOTEBOOK_IMAGES'].split()[0],
+#    os.environ['JUPYTERHUB_IMAGE_SELECTION_NAMES'].split()[1]: os.environ['DOCKER_NOTEBOOK_IMAGES'].split()[1],
+#    os.environ['JUPYTERHUB_IMAGE_SELECTION_NAMES'].split()[2]: os.environ['DOCKER_NOTEBOOK_IMAGES'].split()[2],
+#    os.environ['JUPYTERHUB_IMAGE_SELECTION_NAMES'].split()[3]: os.environ['DOCKER_NOTEBOOK_IMAGES'].split()[3],
+#    'jupyter/scipy-notebook': 'jupyter/scipy-notebook',
+#    'jupyter/r-notebook': 'jupyter/r-notebook',
+#    'jupyter/tensorflow-notebook': 'jupyter/tensorflow-notebook',
+#    'jupyter/datascience-notebook': 'jupyter/datascience-notebook',
+#    'jupyter/pyspark-notebook': 'jupyter/pyspark-notebook',
+#    'jupyter/all-spark-notebook': 'jupyter/all-spark-notebook',
+#}
 #"
 
 # Load jobs to automatically deploy the custom notebooks from the specific images
@@ -311,7 +312,8 @@ export GEOSERVER_ADMIN_PASSWORD=geoserverpass
 # Path to the file containing the clientID for the google drive extension for jupyterlab
 # This file will be mounted into JupyterLab instances.
 # It should contain the following data : {"clientId":"<add_client_id_here>"}
-# To setup a project and find the clientID, check the doc at : https://github.com/jupyterlab/jupyterlab-google-drive/blob/master/docs/setup.md
+# To setup a project and find the clientID, check the doc at :
+#   https://github.com/jupyterlab/jupyterlab-google-drive/blob/master/docs/setup.md
 #export JUPYTER_GOOGLE_DRIVE_SETTINGS=
 
 # URL to terms and conditions for logging into Jupyter.
@@ -332,24 +334,23 @@ export GEOSERVER_ADMIN_PASSWORD=geoserverpass
 #  export JUPYTERHUB_README=""
 #fi
 
+# Timeout (in seconds, default: 3 days) to shut down the user server when no kernels or terminals
+# are running and there is no activity. If undefined or set to zero, the feature will not be enabled.
+#export JUPYTER_IDLE_SERVER_CULL_TIMEOUT=259200
+# Timeout (in seconds, default: 1 day) after which individual
+# user kernels/terminals are considered idle and ready to be culled.
+#export JUPYTER_IDLE_KERNEL_CULL_TIMEOUT=86400
+# Interval (in seconds) on which to check for idle kernels exceeding the cull timeout value.
+# Enabled only if 'JUPYTER_IDLE_KERNEL_CULL_TIMEOUT' is provided and greater than zero.
+# If this value is not provided, equal to zero, or is set higher than 'JUPYTER_IDLE_KERNEL_CULL_TIMEOUT',
+# it will be automatically reduced by half of the timeout value to ensure that it can be effective.
+#export JUPYTER_IDLE_KERNEL_CULL_INTERVAL=0
+
 # Allow for adding new config or override existing config in
 # config/jupyterhub/jupyterhub_config.py.template.
 #
 #export JUPYTERHUB_CONFIG_OVERRIDE="
 #
-# Sample below will shutdown idle server after 3 days and idle kernel after 1 day.
-#
-#c.Spawner.args.extend([
-## Shut down the server after N seconds with no kernels or terminals running and no activity.
-#'--NotebookApp.shutdown_no_activity_timeout={}'.format(3*24*60*60) ,  # 3 days
-## Timeout (in seconds) after which a kernel is considered idle and ready to be culled.
-#'--MappingKernelManager.cull_idle_timeout={}'.format(24*60*60),  # 1 day
-## Culling kernels which have one or more connections for idle but open notebooks.
-## Otherwise, browser have to be closed for culling to work.
-#'--MappingKernelManager.cull_connected=True',
-#])
-#
-#
 # Sample below will allow for sharing notebooks between Jupyter users.
 # Note all shares are public.
 #

docs/source/conf.py

@@ -69,9 +69,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = '1.34.0'
+version = '1.35.0'
 # The full version, including alpha/beta/rc tags.
-release = '1.34.0'
+release = '1.35.0'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.