Skip to content

docs: rewrite of the documentation #219

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 38 commits into from
Mar 13, 2025
Merged
Changes from 6 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
b4235fb
feat: writing error to stderr
cmeesters Jan 22, 2025
666d7d0
docs: describing a few principles and providing some statistics
cmeesters Mar 7, 2025
32a14ea
Update docs/further.md
cmeesters Mar 7, 2025
9bf614c
Update docs/further.md
cmeesters Mar 7, 2025
3301edf
Update docs/further.md
cmeesters Mar 7, 2025
5e844c6
Update docs/further.md
cmeesters Mar 7, 2025
ac75ade
Update docs/further.md
cmeesters Mar 7, 2025
d94f5dc
docs: moved table to new section FAQ
cmeesters Mar 11, 2025
0426bf9
docs: removed references to mamba
cmeesters Mar 11, 2025
0690249
docs: suggesting a lower default latency wait
cmeesters Mar 11, 2025
8bc7dc0
docs: added sample profiles for smp and MPI applications
cmeesters Mar 11, 2025
47f1e51
fix: solve merge conflict
cmeesters Mar 11, 2025
33e2a11
fix: solve merge conflict
cmeesters Mar 11, 2025
6348846
Merge branch 'main' of github.com:snakemake/snakemake-executor-plugin…
cmeesters Mar 11, 2025
d14ad76
Merge branch 'main' into docs/principles
cmeesters Mar 11, 2025
39dc626
Merge branch 'main' of github.com:snakemake/snakemake-executor-plugin…
cmeesters Mar 12, 2025
8d924ec
Merge branch 'main' into docs/principles
cmeesters Mar 12, 2025
867fa93
fix: merge conflicts
cmeesters Mar 12, 2025
c03d0eb
feat: rewriting docs - halfway thru
cmeesters Mar 13, 2025
f7a1955
feat: rewriting docs - done
cmeesters Mar 13, 2025
547f748
fix: typo
cmeesters Mar 13, 2025
3d41db4
fix: link syntax
cmeesters Mar 13, 2025
a79ca6d
fix: erased duplication in line 34
cmeesters Mar 13, 2025
0ba5952
fix: added missing full stop
cmeesters Mar 13, 2025
d24eaa1
fix: typo
cmeesters Mar 13, 2025
9d8b5d1
fix: added missing word
cmeesters Mar 13, 2025
3dc944f
fix: list identation
cmeesters Mar 13, 2025
7d79bcc
fix: typo
cmeesters Mar 13, 2025
7e723dc
fix: wording
cmeesters Mar 13, 2025
3789c91
fix: typo
cmeesters Mar 13, 2025
62136b6
fix: erased word duplication
cmeesters Mar 13, 2025
7a054ed
fix: erased unnessary en-dash
cmeesters Mar 13, 2025
8c42e04
fix: added missing comma
cmeesters Mar 13, 2025
a63e8e1
chore: replace e.g. with "for example", this usually is easier to rea…
dlaehnemann Mar 13, 2025
75e6ced
chore: remove instances of i.e.
dlaehnemann Mar 13, 2025
a6e7702
chore: put each sentence on a separate line for better future diffs
dlaehnemann Mar 13, 2025
afae6ff
chore: one more linebreak
dlaehnemann Mar 13, 2025
ba757b4
Merge branch 'main' into docs/principles
dlaehnemann Mar 13, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 20 additions & 2 deletions docs/further.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,24 @@
### The general Idea
### How this Plugin works

To use this plugin, log in to your cluster's head node (sometimes called the "login" node), activate your environment as usual, and start Snakemake. Snakemake will then submit your jobs as cluster jobs.
Like with all remote executors, Snakemake will essentially submit itself. Specifically, on an HPC cluster Snakemake will be the "job script". Hence, the SLURM logfile will contain the same output, you will otherwise see for that rule. The plugin will consider this as redundant and delete this log file for successful jobs - after all, we have the rule specific logs, too.

Remote executors submit Snakemake jobs, as Snakemake needs to provide its unique functionality (piped group jobs, running rules with wrappers, etc.) on the cluster nodes. Its memory footprint will depend on that added functionality (i.e. for rules with a `run` directive, which imports modules and reads data, it will be bigger).

#### Usage Hints

To use this plugin, you can install it into your Snakemake base environment with conda. It will install its dependency, the "jobstep" plugin (which will be used on the cluster nodes). We recommend installing the `snakemake-storage-plugin-fs` (see below) for automatic stage-in and -out procedures.

We recommend running Snakemake on the login node. Occasionally, HPC administrators are opposed to having a job shepherd running on the login node, since computational tasks should not be executed there.
We therefore provide this table of measurements:

| Workflow | Number of local rules | Total Runtime (hh:mm:ss) | CPU-Time on the login node [user + system] (s) | Fraction |
|:------------- |:---------------|:-------------|:-------------|:-------------:|
| [Transcriptome DiffExp + Fusion detection](https://github.com/snakemake-workflows/transcriptome-differential-expression) | 4 | 9:15:43 | 225.15 | 0.68 |
If you want to contribute similar statistics, please run `/usr/bin/time -v snakemake ...` on your cluster and submit your stats as an [issue to the plugin repo on GitHub](https://github.com/snakemake/snakemake-executor-plugin-slurm/issue).

#### Reporting Bugs and Feature Requests

We welcome bug reports and feature requests! Please report only those bugs related to _this_ plugin as an [issue to the plugin repo on GitHub](https://github.com/snakemake/snakemake-executor-plugin-slurm/issue). All other issues should be directed either to the [Snakemake main repository](https://github.com/snakemake/snakemake/issues) or to the appropriate Snakemake plugin repository (or, if the issue is cluster‑related, to your cluster administrator).

### Specifying Account and Partition

Expand Down