Add in IRIDA Next updates Pipeline Best Practices

[DarianHole]

Added

• nf-schema plugin and associated functions
  • Schemas
  • Param summary, param help, version
  • samplesheetToList
• params.input <CSV> to allow input samplesheets
• iridanext plugin
• nf-prov plugin
• Required nf-core files
• CI tests and linting

Changed

• Final quality metrics output is a CSV now to work with IRIDA next
• Logic for input data
• Logic for skipping specific modules
  • Allowed to skip el_gato ST
  • Allowed to skip el_gato allele plotting
• All process publishDir now in the modules.conf file
• Container for allele plotting
• Adjusted default warn and fail parameters for quality module based on testing
  • min_reads to 60,000 from 150,000

Updated

• Usage and README docs for the input adjustments

Issues Addressed

• Solves Samplesheet input and validation #5
• Solves Add parameter schema #6
• Solves Test data and nf-test #4
• Solves Issue with PLOT_PYSAMSTATS_TSV when running with docker #9

[apetkau]

This is amazing work. Thanks so much Darian 😄 . You've done a great job with this.

I have some in-line comments for you. Some of these are more meant for later PRs, or just recommendations or comments for your information for later. Please let me know if you have any questions.

[apetkau]

Just as a note, there are some additional changes we will need to make for identifiers if you wish to use the sample names in IRIDA Next. But, I would recommend we leave this for a later PR, and even a later release to make things simpler.

As the pipeline is right now, this will work in IRIDA Next, but will use the IRIDA Next assigned identifiers (like INXT_1234) as the sample name instead of user-provided names (like 08-5578).

[apetkau]

This can be for later, but I would recommend a bit of change in logic here for integration with IRIDA Next. Specifically, I'd recommend leaving this to a default of empty and in the pipeline, if empty, using the default data/SeqSphere_1521_schema that comes with the pipeline code.

That way, in IRIDA Next, we can always override it later on if it needs to be updated, but otherwise will default to the database distributed with the pipeline.

[apetkau]

Thanks so much for including this test 😄

In general, I recommend not using stub runs for testing, but instead creating very small inputs and databases and running the full pipeline including all underlying dependency software.

However, doing that all requires time. This is a good first step and later on a full test can be added in here.

[emarinier]

Yeah, echoing this. Testing the stub run only tests that the stub part works, not that the pipeline works under normal circumstances.

[DarianHole]

Yeah I thought that it may be a good spot to put it as a test but makes sense not to. I've changed it to actual data

[emarinier]

Yeah, echoing this. Testing the stub run only tests that the stub part works, not that the pipeline works under normal circumstances.