Skip to content

Commit

Permalink
fix conda and singularity instructions
Browse files Browse the repository at this point in the history
  • Loading branch information
tavareshugo committed Mar 21, 2024
1 parent 593cf5b commit 671c828
Show file tree
Hide file tree
Showing 2 changed files with 29 additions and 12 deletions.
20 changes: 8 additions & 12 deletions materials/04-software.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ title: "Software Management"
:::


## Using pre-installed software
## Using pre-installed software {#sec-module}

It is very often the case that HPC admins have pre-installed several software packages that are regularly used by their users.
Because there can be a large number of packages (and often different versions of the same program), you need to load the programs you want to use in your script using the `module` tool.
Expand Down Expand Up @@ -121,12 +121,13 @@ This means you will use fewer resources and your jobs will complete faster.
Before you use _Mamba_, you will need to install it on the HPC.
If you are attending our live course, we already have _Mamba_ installed, so you can skip this step.

To install _Mamba_, run the following commands from the terminal:
To install _Mamba_, run the following commands from the terminal (this will install it in its default location in the home directory. See @sec-software-csd3 for Cambridge HPC):

```bash
wget "https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-$(uname)-$(uname -m).sh"
bash Mambaforge-$(uname)-$(uname -m).sh -b
rm Mambaforge-$(uname)-$(uname -m).sh
wget "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
bash Miniforge3-$(uname)-$(uname -m).sh -b -p $HOME/miniforge3
rm Miniforge3-$(uname)-$(uname -m).sh
$HOME/miniforge3/bin/mamba init
```

Logout of the HPC and login again, to restart your terminal.
Expand Down Expand Up @@ -416,13 +417,8 @@ Singularity is specifically designed for use in HPC environments and can run on

Typically, Singularity is pre-installed on HPC servers by the system administrators, and **we recommend that you use the version installed by your system admins**.

Alternatively, it is also possible to install Singularity using _Mamba_:

```bash
mamba create -n singularity -c conda-forge singularity
```

However, we have found this to be a less reliable way to setup _Singularity_ on a HPC, as it may require further configuration to interact with the filesystem (in particular as we submit jobs to SLURM).
Although it is possible to install it yourself (e.g. with Mamba), we have found this to be a less reliable way to setup _Singularity_ on a HPC.
This is because it requires further configuration to interact with the filesystem (in particular as we submit jobs to SLURM).


### Singularity images
Expand Down
21 changes: 21 additions & 0 deletions materials/appendices/csd3.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,27 @@ You can see how much space you are using on your storage partitions using the co
-->


## Software {#sec-software-csd3}

There are several **software packages pre-installed** on the HPC and available through the `module` command, as covered in @sec-module.
However, the latest versions of software are not always available.
You can request the HPC helpdesk to install newer versions (or an entirely new software), although they may sometimes not do so, if it's a fast-changing software.

Alternatively, we recommend that you **manage local software using _Mamba_**.
However, due to some software packages being quite large, we recommend you install _Mamba_ on your RDS `rds/hpc-work/` directory (which is 1TB), rather than the home directory (only 40GB).
Here are the instructions to do this:

```bash
wget "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
bash Miniforge3-$(uname)-$(uname -m).sh -b -p $HOME/rds/hpc-work/miniforge3
rm Miniforge3-$(uname)-$(uname -m).sh
$HOME/rds/hpc-work/miniforge3/bin/mamba init
```

Finally, you can use **containers with Singularity**, which is pre-installed on the HPC (no need to load anything).
Do not install your own _Singularity_ (e.g. via Mamba), as it will not be correctly configured for the HPC filesystem.


## Running Jobs

There are two types of nodes that you can access on CSD3:
Expand Down

0 comments on commit 671c828

Please sign in to comment.