Skip to content

Commit d0687fe

Browse files
committed
Fix multiple issues with the CI/CD checks:
There were some breaking changes to the OS images used by GitHub actions, so we have to incorporate corresponding fixes to our actions: * The conda command is not available on all of our CI/CD images anymore, so we don't print the version of it used. * The JDK version in the images was upgraded past the latest version we support (17), so pinned the version to the older one. * The sbt command line tool is no longer included in the images, so we have to manually install it. Additionally, both jupyterlab/maintainer-tools and jupyter-server dropped support for Python 3.8 late last year. Since we have both of those as dependencies, this change also removes Python 3.8 from the test matrix for gateway_provisioners. Finally, this change also fixes multiple lint failures: * Restoring an accidentally missed lint directive * Reformatting the markdown docs with the latest version of mdformat * Upgrading the pre-commit.ci config to use the latest mdformat version
1 parent 700090a commit d0687fe

File tree

20 files changed

+116
-114
lines changed

20 files changed

+116
-114
lines changed

.github/workflows/build.yml

Lines changed: 11 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -12,12 +12,10 @@ jobs:
1212
fail-fast: false
1313
matrix:
1414
os: [ubuntu-latest, macos-latest]
15-
python-version: ["3.8", "3.11"]
15+
python-version: ["3.9", "3.11", "3.12"]
1616
include:
1717
- os: ubuntu-latest
18-
python-version: "3.9"
19-
- os: ubuntu-latest
20-
python-version: "pypy-3.8"
18+
python-version: "pypy-3.9"
2119
- os: macos-latest
2220
python-version: "3.10"
2321
steps:
@@ -26,15 +24,17 @@ jobs:
2624
with:
2725
clean: true
2826
- uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
27+
- uses: actions/setup-java@v4
28+
with:
29+
distribution: temurin
30+
java-version: 17
31+
- uses: sbt/setup-sbt@v1
2932
- name: Display dependency info
3033
run: |
3134
python --version
3235
pip --version
33-
conda --version
34-
- name: Add SBT launcher
35-
run: |
36-
mkdir -p $HOME/.sbt/launchers/1.3.12
37-
curl -L -o $HOME/.sbt/launchers/1.3.12/sbt-launch.jar https://repo1.maven.org/maven2/org/scala-sbt/sbt-launch/1.3.12/sbt-launch.jar
36+
java --version
37+
sbt --version
3838
- name: Install Python dependencies
3939
run: |
4040
pip install ".[dev]"
@@ -61,6 +61,7 @@ jobs:
6161
steps:
6262
- name: Checkout
6363
uses: actions/checkout@v4
64+
- uses: sbt/setup-sbt@v1
6465
- name: Base Setup
6566
uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
6667
- name: Check Release
@@ -76,7 +77,7 @@ jobs:
7677
- uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
7778
- uses: jupyterlab/maintainer-tools/.github/actions/check-links@v1
7879
with:
79-
ignore_links: "http://my-gateway-server.com:8888"
80+
ignore_links: "http://my-gateway-server.com:8888 https://www.gnu.org/software/make/"
8081
ignore_glob: "gateway_provisioners/app-support/README.md"
8182

8283
build_docs:

.pre-commit-config.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ repos:
2727
- id: check-github-workflows
2828

2929
- repo: https://github.com/executablebooks/mdformat
30-
rev: 0.7.17
30+
rev: 0.7.22
3131
hooks:
3232
- id: mdformat
3333
additional_dependencies:

CHANGELOG.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -20,11 +20,11 @@
2020

2121
- Update dependabot config [#130](https://github.com/jupyter-server/gateway_provisioners/pull/130) ([@blink1073](https://github.com/blink1073))
2222
- Update Release Workflows [#129](https://github.com/jupyter-server/gateway_provisioners/pull/129) ([@blink1073](https://github.com/blink1073))
23-
- Bump black\[jupyter\] from 23.9.1 to 23.11.0 [#119](https://github.com/jupyter-server/gateway_provisioners/pull/119) ([@dependabot](https://github.com/dependabot))
23+
- Bump black[jupyter] from 23.9.1 to 23.11.0 [#119](https://github.com/jupyter-server/gateway_provisioners/pull/119) ([@dependabot](https://github.com/dependabot))
2424
- Bump actions/checkout from 3 to 4 [#106](https://github.com/jupyter-server/gateway_provisioners/pull/106) ([@dependabot](https://github.com/dependabot))
25-
- Bump black\[jupyter\] from 23.7.0 to 23.9.1 [#105](https://github.com/jupyter-server/gateway_provisioners/pull/105) ([@dependabot](https://github.com/dependabot))
25+
- Bump black[jupyter] from 23.7.0 to 23.9.1 [#105](https://github.com/jupyter-server/gateway_provisioners/pull/105) ([@dependabot](https://github.com/dependabot))
2626
- Adopt sp-repo-review [#104](https://github.com/jupyter-server/gateway_provisioners/pull/104) ([@blink1073](https://github.com/blink1073))
27-
- Bump black\[jupyter\] from 23.3.0 to 23.7.0 [#99](https://github.com/jupyter-server/gateway_provisioners/pull/99) ([@dependabot](https://github.com/dependabot))
27+
- Bump black[jupyter] from 23.3.0 to 23.7.0 [#99](https://github.com/jupyter-server/gateway_provisioners/pull/99) ([@dependabot](https://github.com/dependabot))
2828
- Update mistune requirement from \<3.0.0 to \<4.0.0 [#94](https://github.com/jupyter-server/gateway_provisioners/pull/94) ([@dependabot](https://github.com/dependabot))
2929
- Bump ruff from 0.0.269 to 0.0.270 [#92](https://github.com/jupyter-server/gateway_provisioners/pull/92) ([@dependabot](https://github.com/dependabot))
3030
- Bump ruff from 0.0.267 to 0.0.269 [#91](https://github.com/jupyter-server/gateway_provisioners/pull/91) ([@dependabot](https://github.com/dependabot))
@@ -63,11 +63,11 @@
6363

6464
- Bump ruff from 0.0.260 to 0.0.261 [#79](https://github.com/jupyter-server/gateway_provisioners/pull/79) ([@dependabot](https://github.com/dependabot))
6565
- Bump ruff from 0.0.259 to 0.0.260 [#77](https://github.com/jupyter-server/gateway_provisioners/pull/77) ([@dependabot](https://github.com/dependabot))
66-
- Bump black\[jupyter\] from 23.1.0 to 23.3.0 [#76](https://github.com/jupyter-server/gateway_provisioners/pull/76) ([@dependabot](https://github.com/dependabot))
66+
- Bump black[jupyter] from 23.1.0 to 23.3.0 [#76](https://github.com/jupyter-server/gateway_provisioners/pull/76) ([@dependabot](https://github.com/dependabot))
6767
- Bump ruff from 0.0.257 to 0.0.259 [#75](https://github.com/jupyter-server/gateway_provisioners/pull/75) ([@dependabot](https://github.com/dependabot))
6868
- Bump ruff from 0.0.254 to 0.0.257 [#73](https://github.com/jupyter-server/gateway_provisioners/pull/73) ([@dependabot](https://github.com/dependabot))
6969
- Bump ruff from 0.0.252 to 0.0.254 [#69](https://github.com/jupyter-server/gateway_provisioners/pull/69) ([@dependabot](https://github.com/dependabot))
70-
- Fix relative link formatting in documentation to be consistent [#68](https://github.com/jupyter-server/gateway_provisioners/pull/68) ([@kiersten-stokes](https://github.com/kiersten-stokes))
70+
- Fix relative link formatting in documentation to be consistent [#68](https://github.com/jupyter-server/gateway_provisioners/pull/68) ([@kiersten-stokes](https://github.com/kiersten-stokes))
7171
- Bump ruff from 0.0.249 to 0.0.252 [#65](https://github.com/jupyter-server/gateway_provisioners/pull/65) ([@dependabot](https://github.com/dependabot))
7272
- Use releaser workflows [#64](https://github.com/jupyter-server/gateway_provisioners/pull/64) ([@blink1073](https://github.com/blink1073))
7373
- Create hatch build env with make-related scripts [#63](https://github.com/jupyter-server/gateway_provisioners/pull/63) ([@kevin-bates](https://github.com/kevin-bates))
@@ -79,7 +79,7 @@
7979
- Add link to SparkOperatorProvisioner class definition [#81](https://github.com/jupyter-server/gateway_provisioners/pull/81) ([@kevin-bates](https://github.com/kevin-bates))
8080
- Fix minor issues in Developer and Contributor docs [#74](https://github.com/jupyter-server/gateway_provisioners/pull/74) ([@kiersten-stokes](https://github.com/kiersten-stokes))
8181
- Fix grammar NITs in Operator's Guide of docs [#72](https://github.com/jupyter-server/gateway_provisioners/pull/72) ([@kiersten-stokes](https://github.com/kiersten-stokes))
82-
- Fix relative link formatting in documentation to be consistent [#68](https://github.com/jupyter-server/gateway_provisioners/pull/68) ([@kiersten-stokes](https://github.com/kiersten-stokes))
82+
- Fix relative link formatting in documentation to be consistent [#68](https://github.com/jupyter-server/gateway_provisioners/pull/68) ([@kiersten-stokes](https://github.com/kiersten-stokes))
8383
- Fix minor errors in Users subsection of documentation [#67](https://github.com/jupyter-server/gateway_provisioners/pull/67) ([@kiersten-stokes](https://github.com/kiersten-stokes))
8484
- Add application support information for deploying JKG and Lab [#66](https://github.com/jupyter-server/gateway_provisioners/pull/66) ([@kevin-bates](https://github.com/kevin-bates))
8585
- Replace references to gateway-experiments with jupyter-server [#62](https://github.com/jupyter-server/gateway_provisioners/pull/62) ([@kevin-bates](https://github.com/kevin-bates))

docs/source/contributors/devinstall.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ will not occur! Always use `make dist` to build the distribution.
2323
### `sbt`
2424

2525
Our Scala launcher is built using `sbt`
26-
([Scala Build Tool](https://www.scala-sbt.org/index.html)). Please check
26+
([Scala Build Tool](https://www.scala-sbt.org/index.html)). Please check
2727
[here](https://www.scala-sbt.org/1.x/docs/Setup.html) for installation instructions for your platform.
2828

2929
## Clone the repo

docs/source/contributors/system-architecture.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -51,8 +51,8 @@ step from its implementation.
5151

5252
### Gateway Provisioner Class Hierarchy
5353

54-
The following block diagram depicts the current class hierarchy for the Gateway Provisioners. The blocks with an
55-
`ABC` badge and dashed border indicate abstract base classes. Those light blue blocks come from `jupyter_client`,
54+
The following block diagram depicts the current class hierarchy for the Gateway Provisioners. The blocks with an
55+
`ABC` badge and dashed border indicate abstract base classes. Those light blue blocks come from `jupyter_client`,
5656
while the others reside in Gateway Provisioners.
5757

5858
```{blockdiag}
@@ -157,8 +157,8 @@ kernel provisioner configuration to override the global value, enabling finer-gr
157157

158158
`ContainerProvisionerBase` is an abstract base class that derives from `RemoteProvisionerBase`. It implements all
159159
the methods inherited from `RemoteProvsionerBase` interacting with the container API and requiring method implementations
160-
to perform the platform's integration. Subclasses
161-
of `ContainerProvisionerBase` must also implement `get_initial_states()`, `get_error_states()`, `get_container_status()`,
160+
to perform the platform's integration. Subclasses
161+
of `ContainerProvisionerBase` must also implement `get_initial_states()`, `get_error_states()`, `get_container_status()`,
162162
and `terminate_container_resources()`:
163163

164164
```python
@@ -204,9 +204,9 @@ manages kernels via a custom resource definition (CRD). For example, `SparkAppli
204204
many components of a Spark-on-Kubernetes application.
205205

206206
`CustomResourceProvisioner` could be considered a _virtual abstract base class_ that provides the necessary method overrides of
207-
`KubernetesProvisioner` to manage the lifecycle of CRDs. If you are going to extend `CustomResourceProvisioner`,
207+
`KubernetesProvisioner` to manage the lifecycle of CRDs. If you are going to extend `CustomResourceProvisioner`,
208208
all that should be necessary is to override these custom resource related attributes (i.e. `group`, `version`, `plural` and
209-
`object_kind`) that define the CRD attributes and its implementation should cover the rest. Note that `object_kind` is
209+
`object_kind`) that define the CRD attributes and its implementation should cover the rest. Note that `object_kind` is
210210
an internal attribute that Gateway Provisioners uses, while the other attributes are associated with the Kubernetes CRD
211211
object definition.
212212

@@ -218,7 +218,7 @@ to function. In addition, the class itself doesn't define any abstract methods
218218

219219
#### `SparkOperatorProvisioner`
220220

221-
A great example of a `CustomResourceProvisioner` is `SparkOperatorProvisioner`. As described in the previous section,
221+
A great example of a `CustomResourceProvisioner` is `SparkOperatorProvisioner`. As described in the previous section,
222222
it's implementation consists of overrides of attributes `group` (e.g, `"sparkoperator.k8s.io"`), `version`
223223
(i.e., `"v1beta2"`), `plural` (i.e., `"sparkapplications"`) and `object_kind` (i.e., `"SparkApplication"`).
224224

@@ -231,7 +231,7 @@ Operators Guide for details.
231231

232232
Gateway Provisioners provides an implementation of a kernel provisioner that communicates with the Docker Swarm resource
233233
manager via the Docker API. When used, the kernels are launched as swarm services and can reside anywhere in the
234-
managed cluster. The core of a Docker Swarm service is a container, so `DockerSwarmProvisioner` derives from
234+
managed cluster. The core of a Docker Swarm service is a container, so `DockerSwarmProvisioner` derives from
235235
`ContainerProvisionerBase`. To leverage kernels configured in this manner, the host application can be deployed either
236236
as a Docker Swarm _service_ or a traditional Docker container.
237237

docs/source/developers/custom-images.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -108,8 +108,8 @@ the appropriate directory in place. For the purposes of this discussion, we'll a
108108
directory, `/usr/local/share/jupyter/kernels`, is externally mounted.
109109

110110
Depending on the environment, Kubernetes or Docker, you can use with `jupyter-k8s-spec` or `jupyter-docker-spec`,
111-
respectively. Invoke the appropriate script by adding the `--image-name` parameter identifying the name of your
112-
custom kernel image. For example, if your custom image is named `acme/data-sci-py:2.0` and you are targeting
111+
respectively. Invoke the appropriate script by adding the `--image-name` parameter identifying the name of your
112+
custom kernel image. For example, if your custom image is named `acme/data-sci-py:2.0` and you are targeting
113113
Kubernetes, issue:
114114

115115
```dockerfile

docs/source/operators/config-file.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ c.DistributedProvisioner.remote_hosts = ["localhost"]
2121

2222
## Provisioner-specific Configuration Options
2323

24-
A complete set of configuration options available for each Gateway Provisioner follows. Where applicable, the
24+
A complete set of configuration options available for each Gateway Provisioner follows. Where applicable, the
2525
configurable option's default value is also provided.
2626

2727
### `KubernetesProvisioner`

docs/source/operators/deploy-distributed.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ Steps required to complete deployment on a distributed cluster are:
1818

1919
## Prerequisites
2020

21-
The distributed capabilities of the `DistributedProvisioner` utilize SSH. As a result, you must ensure appropriate
21+
The distributed capabilities of the `DistributedProvisioner` utilize SSH. As a result, you must ensure appropriate
2222
password-less functionality is in place.
2323

2424
If you want to use Spark in "client mode", you'll want to ensure the `SPARK_HOME` environment variable is properly
@@ -92,9 +92,9 @@ where each provides the following function:
9292
its display name (`display_name`) and language (`language`), as
9393
well as its kernel provisioner's configuration (`metadata.kernel_provisioner`) - which, in this case, will reflect the
9494
`DistributedProvisioner`.
95-
- `logo-64x64.png` - the icon resource corresponding to this kernel specification. Icon resource files must be start
95+
- `logo-64x64.png` - the icon resource corresponding to this kernel specification. Icon resource files must be start
9696
with the `logo-` prefix to be included in the kernel specification.
97-
- `scripts/launch_ipykernel.py` - the "launcher" for the IPyKernel kernel (or subclasses thereof). This file is typically
97+
- `scripts/launch_ipykernel.py` - the "launcher" for the IPyKernel kernel (or subclasses thereof). This file is typically
9898
implemented in the language of the kernel and is responsible for creating the local connection information, asynchronously
9999
starting a SparkContext (if asked), spawning a listener process to receive interrupts and shutdown requests, and starting
100100
the IPyKernel itself.
@@ -300,7 +300,7 @@ To see all available configurables, use `--help-all`.
300300
## Specifying a load-balancing algorithm
301301

302302
The `DistributedProvisioner` provides two ways to configure how kernels are distributed across
303-
the configured set of hosts: round-robin or least-connection. This configurable option is a _host application_
303+
the configured set of hosts: round-robin or least-connection. This configurable option is a _host application_
304304
setting and is not available to be overridden on a per-kernel basis.
305305

306306
### Round-robin
@@ -362,7 +362,7 @@ YARN client mode kernel specifications can be considered _distributed mode kerne
362362
happen to use `spark-submit` from different nodes in the cluster but use the
363363
`DistributedProvisioner` to manage their lifecycle.
364364

365-
These kernel specifications are generated using the `'--spark'` command line option as noted above. When provided,
365+
These kernel specifications are generated using the `'--spark'` command line option as noted above. When provided,
366366
a kernel specification similar to the following is produced:
367367

368368
```json

docs/source/operators/deploy-docker.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -16,10 +16,10 @@ for examples of how to configure and deploy such applications.
1616

1717
## Generating Kernel Specifications
1818

19-
Kernelspec generation for Docker and Docker Swarm deployments is performed using the `jupyter-docker-spec` command. Because
19+
Kernelspec generation for Docker and Docker Swarm deployments is performed using the `jupyter-docker-spec` command. Because
2020
the host application will also reside within a docker image, the commands are usually placed into a Dockerfile
21-
that _extends_ an existing image. However, some may choose to `docker exec` into a running container, perform and test
22-
the necessary configuration, then use `docker commit` to generate a new image. That said, the following will assume a
21+
that _extends_ an existing image. However, some may choose to `docker exec` into a running container, perform and test
22+
the necessary configuration, then use `docker commit` to generate a new image. That said, the following will assume a
2323
Dockerfile approach.
2424

2525
```{attention}
@@ -57,10 +57,10 @@ where each provides the following function:
5757
its display name (`display_name`) and language (`language`), as
5858
well as its kernel provisioner's configuration (`metadata.kernel_provisioner`) - which, in this case, will reflect the
5959
`DockerProvisioner`.
60-
- `logo-64x64.png` - the icon resource corresponding to this kernel specification. Icon resource files must be start
60+
- `logo-64x64.png` - the icon resource corresponding to this kernel specification. Icon resource files must be start
6161
with the `logo-` prefix to be included in the kernel specification.
6262
- `scripts/launch_docker.py` - the "launcher" for the kernel image identified by the
63-
`metadata.kernel_provisioner.config.image_name` entry. This file can be modified to include instructions for
63+
`metadata.kernel_provisioner.config.image_name` entry. This file can be modified to include instructions for
6464
volume mounts, etc., and is compatible with both Docker and Docker Swarm - performing the applicable instructions for
6565
each environment.
6666

@@ -71,7 +71,7 @@ others.
7171

7272
### Generating Multiple Specifications
7373

74-
Its common practice to support multiple languages or use different images for kernels of the same language. For each
74+
Its common practice to support multiple languages or use different images for kernels of the same language. For each
7575
of those differences, a separate installation command should be provided:
7676

7777
```dockerfile
@@ -102,9 +102,9 @@ Items worth noting:
102102
## Other Configuration Items
103103

104104
There are some environment variables that can be set in the host application's environment that affect how Gateway
105-
Provisioners operate within a Docker and Docker Swarm environment. For example, `GP_MIRROR_WORKING_DIRS` can be set
105+
Provisioners operate within a Docker and Docker Swarm environment. For example, `GP_MIRROR_WORKING_DIRS` can be set
106106
to `True`, instructing Gateway Provisioners to set the launched container's working directory to the value of
107-
`KERNEL_WORKING_DIR`. When this environment variable is enabled, it usually implies that volume mounts are in play
107+
`KERNEL_WORKING_DIR`. When this environment variable is enabled, it usually implies that volume mounts are in play
108108
such that the per-user volumes are then available to the launched container.
109109

110110
Other [environment variables](config-add-env.md#additional-environment-variables) applicable to Docker/Docker Swarm
@@ -216,7 +216,7 @@ To see all configurable class-options for some <cmd>, use:
216216
Examples
217217
--------
218218
219-
jupyter-docker-spec install --language=R --kernel-name=r_docker --image_name=foo/my_r_kernel_image:v4_0
219+
jupyter-docker-spec install --language=R --kernel-name=r_docker --image-name=foo/my_r_kernel_image:v4_0
220220
221221
jupyter-docker-spec install --swarm --kernel-name=python_swarm
222222

0 commit comments

Comments
 (0)