Release workflow for new cult-cargo versions

[o-smirnov]

@SpheMakh @Athanaseus see also #8.

By necessity, we have a somewhat parallel release process. The cult-cargo package is uploaded to PyPI, and a corresponding set of images needs to be pushed to quay.io/stimela2.

Checklist for a new release:

• start a branch on cult-cargo for the release.

• update version number in https://github.com/caracal-pipeline/cult-cargo/blob/master/pyproject.toml. This is the PyPI release version.

• update version number in https://github.com/caracal-pipeline/cult-cargo/blob/master/cultcargo/genesis/cult-cargo-base.yml. This is the version that stimela will look for when downloading images.

• update version number in https://github.com/caracal-pipeline/cult-cargo/blob/master/cultcargo/builder/cargo-manifest.yml. This is the version number that the build-cargo.py script will use to tag images.

• check stimela version in https://github.com/caracal-pipeline/cult-cargo/blob/master/cultcargo/images/python-astro/Dockerfile. The only real reason to install stimela in this image is to provide scabha.schema_utils to interested packages (which use python-astro as a base image). So I think this should be the same version as listed in the pyproject.toml dependency.

• run build-cargo.py -a to rebuild and push all images with the new image version tags, and make sure this completes successfully.

• open a PR and ask for a review.

It's a bit of a hassle having to update the version number in three places, but I haven't figured out an elegant way to automate this.

Also note that the image versions do not necessarily have to match the PyPI package version. We could still choose to follow a scheme (see #8) where image versions are only bumped when existing images change. So we could, for example, carry on using images versioned cc-0.2.0 (or even just cc-0.2) with patch releases 0.2.0, 0.2.1, etc. I am undecided.

Pros of decoupling image versions from PyPI package version:

• minor tweaks to cab schemas can be pushed out as patch release without needing to rebuild images.

Cons of decoupling:

• confusion!

Also, once a release of cult-cargo is on PyPI, one thing we should NEVER do is push out new images under the same image version tag. That will definitely mess with reproducibility. As long as the release is still in "candidate" stage (i.e. lives on a branch and hasn't been pushed to PyPI), we can build and rebuild and push images to our heart's content, but once the release is public, those images mustn't change again. The current workflow has no provisions to protect against this apart from self-discipline. I wonder if we can't build in protections into the build-cargo.py script? I see quay.io provides an option to make an image tag immutable, but this has to be done via the web interface, image by image, which is obviously not practical. I don't see a standard API for doing this programmatically -- seems to be registry-specific, anyway, and there's nothing obvious on the quay.io API docs.

[o-smirnov]

The current workflow has no provisions to protect against this apart from self-discipline. I wonder if we can't build in protections into the build-cargo.py script?

Actually, this is not difficult. E.g. docker inspect quay.io/stimela2/python-astro will return all sorts of information about the images available, including tags. So we can simply refuse to push a new image if the tag already exists. Or even smarter, if the image version ends with "rc_X_", allow the push, otherwise not. That way, when you're doing development on your own branch, you can change the image versions to e.g. "0.2.0.rc_X_", and push development images without running into the protection.

[o-smirnov]

So maybe the following logic for the builder push:

• Get cult-cargo package version via importlib.

• Get image tag from cargo manifest.

• (Get image tag from genesis/cult-cargo-base.yml and check that this matches, bail out if not)

• (Warn if image tag and package version are different -- depends if we allow version decoupling?)

• Check github releases. This tells us if the working cult-cargo package version is released, or is in testing.

• Check if images with the current tag are already pushed. See also docker inspect API.

If images haven't been pushed, allow pushing.

If images have already been pushed:

• If image tag doesn't match package version: big error message, and refuse to push. This can only arise if we support version decoupling, anyway. And we should never be pushing to a (presumably, older) image tag.

• If package is released, and is not an "rc" version: refuse to push.

• If package is released, and is an "rc" version, or package unreleased: ok to push. Presumably, we're testing and building a new version, so image churn is allowed.

We can delete images by hand via the website to work around this, if ever needed to recover from some kind of mess-up.

[o-smirnov]

Simplified checklist, as of #11 and 0.1.2rc1:

• start a branch on cult-cargo for the release.

• update version number in https://github.com/caracal-pipeline/cult-cargo/blob/master/pyproject.toml. This is the PyPI release version. This can be set to "rcX" for release candidates.

• update version number in https://github.com/caracal-pipeline/cult-cargo/blob/master/cultcargo/genesis/cult-cargo-base.yml. This does not need to (but can) include rc tags.

• check stimela version in https://github.com/caracal-pipeline/cult-cargo/blob/master/cultcargo/images/python-astro/Dockerfile. The only real reason to install stimela in this image is to provide scabha.schema_utils to interested packages (which use python-astro as a base image). So I think this should be the same version as listed in the pyproject.toml dependency.

• run build-cargo.py -a to rebuild and push all images with the new image version tags, and make sure this completes successfully.

• open a PR and ask for a review.