This repository was archived by the owner on Feb 26, 2025. It is now read-only.
deploy: merge upstream Spack v0.21 release branch #2213
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Update DataTransferKit for 3.1.1 release * Require Trilinos-14 for 3.1.0 and higher
* ldak: update to 5.2, add maintainer * ldak: use compiler.openmp_flag
* itk: patch missing include for newer compilers * itk: The package doesn't use MPI * itk: package requires the high-level hdf5 api * itk: patch url with ?full_index=1 * itk: point to 4041 commit in master * itk: don't constrain hdf5 with ~mpi
Co-authored-by: Nicolas Richart <nrichart@users.noreply.github.com>
* Fix cmake_args for spglib v2.1.0+ * Add option to build fortran interface in package spglib * fix style as sugested by ci/prechecks/style * Enable fortran variant from v1.16.4 as suggested Co-authored-by: Rocco Meli <r.meli@bluemail.ch> --------- Co-authored-by: Rocco Meli <r.meli@bluemail.ch>
otherwise it detected pre-configured compilers in an potentially different way.
* gromacs: default to external blas & lapack * drop vendored lapack/blas altogether
Credits to @ChristianKniep for advocating the idea of OCI image layers being identical to spack buildcache tarballs. With this you can configure an OCI registry as a buildcache: ```console $ spack mirror add my_registry oci://user/image # Dockerhub $ spack mirror add my_registry oci://ghcr.io/haampie/spack-test # GHCR $ spack mirror set --push --oci-username ... --oci-password ... my_registry # set login credentials ``` which should result in this config: ```yaml mirrors: my_registry: url: oci://ghcr.io/haampie/spack-test push: access_pair: [<username>, <password>] ``` It can be used like any other registry ``` spack buildcache push my_registry [specs...] ``` It will upload the Spack tarballs in parallel, as well as manifest + config files s.t. the binaries are compatible with `docker pull` or `skopeo copy`. In fact, a base image can be added to get a _runnable_ image: ```console $ spack buildcache push --base-image ubuntu:23.04 my_registry python Pushed ... as [image]:python-3.11.2-65txfcpqbmpawclvtasuog4yzmxwaoia.spack $ docker run --rm -it [image]:python-3.11.2-65txfcpqbmpawclvtasuog4yzmxwaoia.spack ``` which should really be a game changer for sharing binaries. Further, all content-addressable blobs that are downloaded and verified will be cached in Spack's download cache. This should make repeated `push` commands faster, as well as `push` followed by a separate `update-index` command. An end to end example of how to use this in Github Actions is here: **https://github.com/haampie/spack-oci-buildcache-example** TODO: - [x] Generate environment modifications in config so PATH is set up - [x] Enrich config with Spack's `spec` json (this is allowed in the OCI specification) - [x] When ^ is done, add logic to create an index in say `<image>:index` by fetching all config files (using OCI distribution discovery API) - [x] Add logic to use object storage in an OCI registry in `spack install`. - [x] Make the user pick the base image for generated OCI images. - [x] Update buildcache install logic to deal with absolute paths in tarballs - [x] Merge with `spack buildcache` command - [x] Merge spack#37441 (included here) - [x] Merge spack#39077 (included here) - [x] spack#39187 + spack#39285 - [x] spack#39341 - [x] Not a blocker: spack#35737 fixes correctness run env for the generated container images NOTE: 1. `oci://` is unfortunately taken, so it's being abused in this PR to mean "oci type mirror". `skopeo` uses `docker://` which I'd like to avoid, given that classical docker v1 registries are not supported. 2. this is currently `https`-only, given that basic auth is used to login. I _could_ be convinced to allow http, but I'd prefer not to, given that for a `spack buildcache push` command multiple domains can be involved (auth server, source of base image, destination registry). Right now, no urllib http handler is added, so redirects to https and auth servers with http urls will simply result in a hard failure. CAVEATS: 1. Signing is not implemented in this PR. `gpg --clearsign` is not the nicest solution, since (a) the spec.json is merged into the image config, which must be valid json, and (b) it would be better to sign the manifest (referencing both config/spec file and tarball) using more conventional image signing tools 2. `spack.binary_distribution.push` is not yet implemented for the OCI buildcache, only `spack buildcache push` is. This is because I'd like to always push images + deps to the registry, so that it's `docker pull`-able, whereas in `spack ci` we really wanna push an individual package without its deps to say `pr-xyz`, while its deps reside in some `develop` buildcache. 3. The `push -j ...` flag only works for OCI buildcache, not for others
…indows (spack#39850) .bat or .exe files can be considered executable on Windows. This PR expands the regex for detectable packages to allow for the detection of packages that vendor .bat wrappers (intel mpi for example). Additional changes: * Outside of Windows, when searching for executables `path_hints=None` was used to indicate that default path hints should be provided, and `[]` was taken to mean that no defaults should be chosen (in that case, nothing is searched); behavior on Windows has now been updated to match. * Above logic for handling of `path_hints=[]` has also been extended to library search (for both Linux and Windows). * All exceptions for external packages were documented as timeout errors: this commit adds a distinction for other types of errors in warning messages to the user.
* e4s ci: add exago +cuda, +rocm builds * exago: rename 5-18-2022-snapshot to snapshot.5-18-2022 * disable exago +rocm for non-external rocm ci install * note that hiop +rocm fails to find hip libraries when they are spack-installed
Typically MSVC is detected via the VSWhere program. However, this may not be available, or may be installed in an unpredictable location. This PR adds an additional approach via Windows Registry queries to determine VS install location root. Additionally: * Construct vs_install_paths after class-definition time (move it to variable-access time). * Skip over keys for which a user does not have read permissions when performing searches (previously the presence of these keys would have caused an error, regardless of whether they were needed). * Extend helper functionality with option for regex matching on registry keys vs. exact string matching. * Some internal refactoring: remove boolean parameters in some cases where the function was always called with the same value (e.g. `find_subkey`)
* py-bluepyemodel: new package with dependencies
* py-morphio: add MPI as dependency to avoid failing builds
* Formatting
* py-bluepyefe: no need to set NEURON_INIT_MPI
* py-morphio: unifurcation branch is ancient history
* py-bluepyopt: only set NEURON_INIT_MPI with +neuron
* py-efel: get rid of old version
* py-morph{-tool,io}: rename develop to master to match branch
* py-bluepyefe: unset PMI_RANK is also neuron-related
* py-bluepyopt: PMI_RANK is also neuron-related
* Implement review remarks
* py-morph-tool, py-neurom: small fixes
* py-morphio: reword dependencies
Co-authored-by: Benjamin Meyers <bsmits@rit.edu>
* [add] py-generateds: new package * py-generateds: Update from review Co-authored-by: Manuela Kuhn <36827019+manuelakuhn@users.noreply.github.com> * py-generateds: add versions 2.41.5, 2.42.1, 2.42.2, 2.43.1 and 2.43.2 --------- Co-authored-by: LydDeb <lyderic.debusschere@eolen.com> Co-authored-by: Manuela Kuhn <36827019+manuelakuhn@users.noreply.github.com>
* [add] py-moarchiving: new package * py-moarchiving: update from review: description, variant default value is False, switch when and type --------- Co-authored-by: LydDeb <lyderic.debusschere@eolen.com>
* Add liggght patched for newer compiler Add C++ 17 support Add Clang and Oneapi support * Add maintainers * Fix format in liggghts * Fix maintainers before versions Co-authored-by: Alec Scott <alec@bcs.sh> * Fix style and user to usr * Update package.py --------- Co-authored-by: Alec Scott <alec@bcs.sh>
* acts: new version from 28.1.0 to 30.3.1 * acts: new version 30.3.2 * acts: new variant +podio
…k#40720) The code should be functonally equivalent to what it was before, but now to avoid cycles by design we are using a "hidden" feature of clingo
|
To test your PR, use the following on BlueBrain5: unset MODULEPATH
. /gpfs/bbp.cscs.ch/ssd/apps/bsd/pulls/2213/config/modules.sh
module load unstable |
|
The following build jobs of the deployment for this PR failed: Please fix the following packages that failed to build: |
|
The following build jobs of the deployment for this PR failed: The following module tests failed: |
|
The following build jobs of the deployment for this PR failed: The following module tests failed: |
|
The following build jobs of the deployment for this PR failed: The following module tests failed: |
|
The following build jobs of the deployment for this PR failed: The following module tests failed: |
|
The following build jobs of the deployment for this PR failed: The following module tests failed: |
|
The following build jobs of the deployment for this PR failed: The following module tests failed: |
|
The following build jobs of the deployment for this PR failed: The following module tests failed: |
|
To test your PR, use the following on BlueBrain5: unset MODULEPATH
. /gpfs/bbp.cscs.ch/ssd/apps/bsd/pulls/2213/config/modules.sh
module load unstablePlease test the following updated modules:
The following module tests failed: |
|
The following build jobs of the deployment for this PR failed: Please fix the following packages that failed to build: |
|
To test your PR, use the following on BlueBrain5: unset MODULEPATH
. /gpfs/bbp.cscs.ch/ssd/apps/bsd/pulls/2213/config/modules.sh
module load unstablePlease test the following updated modules:
The following module tests failed: |
|
To test your PR, use the following on BlueBrain5: unset MODULEPATH
. /gpfs/bbp.cscs.ch/ssd/apps/bsd/pulls/2213/config/modules.sh
module load unstablePlease test the following updated modules:
The following module tests failed: |
|
To test your PR, use the following on BlueBrain5: unset MODULEPATH
. /gpfs/bbp.cscs.ch/ssd/apps/bsd/pulls/2213/config/modules.sh
module load unstablePlease test the following updated modules:
The following module tests failed: |
|
To test your PR, use the following on BlueBrain5: unset MODULEPATH
. /gpfs/bbp.cscs.ch/ssd/apps/bsd/pulls/2213/config/modules.sh
module load unstablePlease test the following updated modules:
The following module tests failed: |
To be replaced by @heerener's work later/soon™
|
To test your PR, use the following on BlueBrain5: unset MODULEPATH
. /gpfs/bbp.cscs.ch/ssd/apps/bsd/pulls/2213/config/modules.sh
module load unstablePlease test the following updated modules:
The following module tests failed: |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.