diff --git a/docs/further.md b/docs/further.md index e6135f80..4632b85c 100644 --- a/docs/further.md +++ b/docs/further.md @@ -286,6 +286,12 @@ set-resources: Be sure to use sensible settings for your cluster and make use of parallel execution (e.g. threads) and [global profiles](#using-profiles) to avoid I/O contention. +## Nesting Jobs (or Running this Plugin within a Job) + +Some environments provide a shell within a SLURM job, for instance, IDEs started in on-demand context. If Snakemake attempts to use this plugin to spawn jobs on the cluster, this may work just as intended. Or it might not: depending on cluster settings or individual settings, submitted jobs may be ill-parameterized or will not find the right environment. + +If the plugin detects to be running within a job, it will therefore issue a warning and stop for 5 seconds. + ## Summary: When put together, a frequent command line looks like: diff --git a/snakemake_executor_plugin_slurm/__init__.py b/snakemake_executor_plugin_slurm/__init__.py index 08ebe1ac..00e3eb24 100644 --- a/snakemake_executor_plugin_slurm/__init__.py +++ b/snakemake_executor_plugin_slurm/__init__.py @@ -50,11 +50,24 @@ # Implementation of your executor class Executor(RemoteExecutor): def __post_init__(self): + # run check whether we are running in a SLURM job context + self.warn_on_jobcontext() self.run_uuid = str(uuid.uuid4()) self.logger.info(f"SLURM run ID: {self.run_uuid}") self._fallback_account_arg = None self._fallback_partition = None + def warn_on_jobcontext(self, done=None): + if not done: + if "SLURM_JOB_ID" in os.environ: + self.logger.warning( + "Running Snakemake in a SLURM within a job may lead" + " to unexpected behavior. Please run Snakemake directly" + " on the head node." + ) + time.sleep(5) + done = True + def additional_general_args(self): return "--executor slurm-jobstep --jobs 1"