invalid choice: 'slurm-jobstep' (choose from 'local', 'd ryrun', 'touch')

[radaniba]

Hello everyone

I am trying to execute a Snakemake pipeline on a slurm cluster but I get this error

"error: argument --executor/-e: invalid choice: 'slurm-jobstep' (choose from 'local', 'dryrun', 'touch')"

I am using the latest version of snakemake and also using an older version (unfortunately cannot upgrade) of slurm (~19)

Locally everything seems to work Ok but when I use executer slurm in my profile this error occure in all my log files (the jobs I mean)

Any idea ?

[cmeesters]

Hi,

I can only assume there is an issue with your setup. The idea is that this plugin is a dependency of the snakemake-executor-plugin-slurm and not installed by itself. So, when you work on a login- or head-node of your cluster, Snakemake essentially submits itself. Only within a job the jobstep plugin is used. So:

• did you install this plugin by installing the snakemake-executor-plugin-slurm?
• are you by any chance working within job context, e.g. by some editor or IDE which shell runs within a job? There is a PR to prevent this (feat: warning if run in job snakemake-executor-plugin-slurm#78), but it's still under review.

If these hints do not help, please post your environment (e.g. by appending a text file). You get it by invoking env. And post your command line, resp. your profile.

[radaniba]

Hello @cmeesters
Yes, snakemake-executor-plugin-slurm is installed and regarding the second point the answer is no, this is a typical bioinformatics pipeline so basically calling tools with default arguments.

My profile looks like this

config.v8+.yaml

executor: slurm
jobs: 10

default-resources:
    slurm_partition: "hannibal_high_mem"
    mem_mb: 23552
    slurm_account: "root"

cluster.yaml

__default__:
  mem: 23552
  threads: 1

The way I run my workflows is like this :

snakemake --profile profile_name

Now I get this weired error

WorkflowError:
The given account root appears to be invalid. Available accounts:

Even though saactmgr is listing the account

[radaniba]

apologies this last comment was fixed but my my main issue with the executor still there

main.py: error: argument --executor/-e: invalid choice: 'slurm-jobstep' (choose from 'local', 'dryrun', 'touch')

The rule looks like this

rule trim_galore:
    input:
        fastq1 = os.path.join(FASTQ_DIR,"{sample}_1.fastq.gz"),
        fastq2 = os.path.join(FASTQ_DIR,"{sample}_2.fastq.gz")
    output:
        trimmed_fastq1 = os.path.join(TRIM_OUTPUT_DIR, "{sample}_val_1.fq.gz"),
        trimmed_fastq2 = os.path.join(TRIM_OUTPUT_DIR, "{sample}_val_2.fq.gz"),
        unpaired_fastq1 = os.path.join(TRIM_OUTPUT_DIR, "{sample}_val_1_unpaired.fq.gz"),
        unpaired_fastq2 = os.path.join(TRIM_OUTPUT_DIR, "{sample}_val_2_unpaired.fq.gz")
    shell:
        """
        trim_galore --paired --retain_unpaired {input.fastq1} {input.fastq2} --output_dir {TRIM_OUTPUT_DIR}
        """
        ```

 

[radaniba]

I apologize, I found my issue, I had to get this plugin setup in all nodes :) silly mistake

[cmeesters]

Glad it's working. Just 2 cents:

• the assumption is, of course, a shared file system, such that you only need to install once.
• 'root' is an unlikely SLURM account. Usually, SLURM derives from 'none' - but users cannot submit using this account. On systems with a default account or no accounting at all, users may omit the slurm_account
• and a third: if you are dealing with different partitions, it's best not to run all in a high memory partition, as indicated by your example.

I really hope this helps.
Cheers
Christian