-
Notifications
You must be signed in to change notification settings - Fork 4
add nf-core style module for assembly_manifest #12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
20 commits
Select commit
Hold shift + click to select a range
ef17f01
add nf-core module template
fxquah 00e7c3b
first draft of the process
fxquah f84bcb1
add wave container
fxquah 7ec537d
drop version write for assembly-uploader for now
fxquah 05c85f6
run pre-commit
fxquah 5964581
pre-commit on assets/samplesheet.csv
fxquah e55b8ad
add fasta as input param, add unit test
fxquah d8d3efa
remove a comment
fxquah 6d1abd8
use nf-core/test-datasets
fxquah 32e0200
fix nf-test file download paths
fxquah 4ac6e27
rename module to generate_assembly_manifest
fxquah eecee92
remove old test data files
fxquah ea37f8b
regenerate snapshot
fxquah 314ce84
make stub test use dummy files that dont exist
fxquah e1e8f1a
update meta.yml
fxquah 16d46ee
change --study to use an input value, explicitly define output even t…
fxquah be5f69d
update changelog
fxquah 540b129
update env yml
fxquah 37b4e18
remove glob
fxquah 0cacda1
make test mroe specific
fxquah File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,3 @@ | ||
| sample,fasta,accession,assembly_software,binning_software,binning_parameters,stats_generation_software,completeness,contamination,genome_coverage,metagenome,co-assembly,broad_environment,local_environment,environmental_medium,rRNA_presence,NCBI_lineage | ||
| lachnospira_eligens,https://github.com/nf-core/test-datasets/raw/seqsubmit/test_data/bins/bin_lachnospira_eligens.fa.gz,SRR24458089,spades_v3.15.5,mags_v1,default,CheckM2_v1.0.1,61.0,0.21,32.07,sediment metagenome,False,marine,cable bacteria,marine sediment,False,d__Bacteria;p__Proteobacteria;c__Deltaproteobacteria;o__Desulfobacterales;f__Desulfobulbaceae;g__Candidatus Electrothrix;s__ | ||
| lachnospiraceae,https://github.com/nf-core/test-datasets/raw/seqsubmit/test_data/bins/bin_lachnospiraceae.fa.gz,SRR24458087,spades_v3.15.5,mags_v1,default,CheckM2_v1.0.1,92.81,1.09,66.04,sediment metagenome,False,marine,cable bacteria,marine sediment,False,d__Bacteria;p__Proteobacteria;c__Deltaproteobacteria;o__Desulfobacterales;f__Desulfobulbaceae;g__Candidatus Electrothrix;s__Candidatus Electrothrix marina | ||
| lachnospiraceae,https://github.com/nf-core/test-datasets/raw/seqsubmit/test_data/bins/bin_lachnospiraceae.fa.gz,SRR24458087,spades_v3.15.5,mags_v1,default,CheckM2_v1.0.1,92.81,1.09,66.04,sediment metagenome,False,marine,cable bacteria,marine sediment,False,d__Bacteria;p__Proteobacteria;c__Deltaproteobacteria;o__Desulfobacterales;f__Desulfobulbaceae;g__Candidatus Electrothrix;s__Candidatus Electrothrix marina |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| --- | ||
| channels: | ||
| - conda-forge | ||
| - bioconda | ||
| dependencies: | ||
| - "bioconda::assembly_uploader=1.3.2" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| process GENERATE_ASSEMBLY_MANIFEST { | ||
| tag "$meta.id" | ||
| label 'process_single' | ||
|
|
||
| container "community.wave.seqera.io/library/pip_assembly-uploader:7e9461afbdd7a521" | ||
|
|
||
| input: | ||
| tuple val(meta), path(assembly_fasta), path(data_csv) | ||
| val(assembly_study) | ||
|
|
||
| output: | ||
| tuple val(meta), path("${assembly_study}_upload/*.manifest") , emit: manifest | ||
| path "versions.yml", emit: versions | ||
|
|
||
| when: | ||
| task.ext.when == null || task.ext.when | ||
|
|
||
| script: | ||
| def args = task.ext.args ?: '' | ||
| def prefix = task.ext.prefix ?: "${meta.id}" | ||
| """ | ||
| assembly_manifest \\ | ||
| --study ${assembly_study} \\ | ||
| --data ${data_csv} \\ | ||
| --assembly_study ${assembly_study} \\ | ||
| --output ${assembly_study}_upload \\ | ||
| ${args} | ||
|
|
||
| cat <<-END_VERSIONS > versions.yml | ||
| "${task.process}": | ||
| python: \$(python --version) | ||
| END_VERSIONS | ||
| """ | ||
|
|
||
| stub: | ||
| def args = task.ext.args ?: '' | ||
| def prefix = task.ext.prefix ?: "${meta.id}" | ||
| """ | ||
| mkdir ${assembly_study}_upload/ | ||
| touch ${assembly_study}_upload/test.manifest | ||
|
|
||
| cat <<-END_VERSIONS > versions.yml | ||
| "${task.process}": | ||
| python: \$(python --version) | ||
| END_VERSIONS | ||
| """ | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| name: "generate_assembly_manifest" | ||
| description: Generate manifest files for genome assemblies using assembly-uploader | ||
| keywords: | ||
| - ena | ||
| - manifest | ||
| - upload | ||
| tools: | ||
| - assembly-uploader: | ||
| description: Make a manifest file for each assembly. | ||
| Co-assemblies cannot be generated from a mix of private and public runs - | ||
| all runs used in a co-assembly must have the same privacy status (all private or all public). | ||
| documentation: https://github.com/EBI-Metagenomics/assembly_uploader/ | ||
| licence: ["Apache License"] | ||
| input: | ||
| - - meta: | ||
| type: map | ||
| description: | | ||
| Groovy Map containing sample information. | ||
| e.g. [ id:'test', single_end:false ] | ||
| - assembly_fasta: | ||
| type: file | ||
| description: Genome assembly FASTA file. | ||
| - data_csv: | ||
| type: file | ||
| description: | | ||
| CSV file containing metadata - | ||
| runs (comma-separated and in quotes, example: "SRR1234,SRR5678"), | ||
| coverage, assembler, version, filepath and optionally sample. | ||
| - assembly_study: | ||
| type: value | ||
| description: | | ||
| Pre-existing study ID to submit to if available. | ||
| Must exist in the webin account. | ||
|
|
||
| output: | ||
| - meta: | ||
| type: map | ||
| description: | | ||
| Groovy Map containing sample information. | ||
| e.g. [ id:'test', single_end:false ] | ||
| - manifest: | ||
| type: file | ||
| description: Manifest file required for submission with webin-cli. | ||
| - versions: | ||
| type: file | ||
| description: File containing software versions. | ||
| pattern: "versions.yml" | ||
|
|
||
| authors: | ||
| - "@fxquah" |
73 changes: 73 additions & 0 deletions
73
modules/local/generate_assembly_manifest/tests/main.nf.test
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| nextflow_process { | ||
|
|
||
| name "Test Process GENERATE_ASSEMBLY_MANIFEST" | ||
| script "../main.nf" | ||
| process "GENERATE_ASSEMBLY_MANIFEST" | ||
|
|
||
| tag "modules" | ||
| tag "modules_" | ||
| tag "generate_assembly_manifest" | ||
|
|
||
| test("GENERATE_ASSEMBLY_MANIFEST completes with expected outputs") { | ||
| when { | ||
| process { | ||
| """ | ||
| input[0] = [ | ||
| [ id:'test' ], | ||
| file(params.pipelines_testdata_base_path + "/test_data/contigs.fasta.gz", checkIfExists: true), | ||
| file(params.pipelines_testdata_base_path + "/samplesheets/samplesheet_generatemanifest.csv", checkIfExists: true) | ||
| ] | ||
| input[1] = "PRJ12345" | ||
| """ | ||
|
|
||
| } | ||
| } | ||
|
|
||
| then { | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please, check for "PRJ12345" in manifest file and .manifest exists in output folder There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. done - test is more specific now |
||
| assert process.success | ||
| assertAll( | ||
| { assert snapshot( | ||
| process.out, | ||
| path(process.out.versions[0]).yaml | ||
| ).match() }, | ||
| { assert process.out.manifest.size() == 1 }, | ||
| { assert process.out.manifest[0][1].toString().endsWith(".manifest") }, | ||
| { | ||
| def manifestContent = path(process.out.manifest[0][1]).text | ||
| assert manifestContent.contains("PRJ12345") : "PRJ12345 not found in manifest file" | ||
| } | ||
| ) | ||
| } | ||
|
|
||
| } | ||
|
|
||
| test("GENERATE_ASSEMBLY_MANIFEST completes with expected outputs - stub") { | ||
|
|
||
| options "-stub" | ||
|
|
||
| when { | ||
| process { | ||
| """ | ||
| input[0] = [ | ||
| [ id:'test' ], | ||
| file('test_contigs.fastq.gz'), | ||
| file('test_samplesheet_generatemanifest.csv') | ||
| ] | ||
| input[1] = "PRJ12345" | ||
| """ | ||
| } | ||
| } | ||
|
|
||
| then { | ||
| assert process.success | ||
| assertAll( | ||
| { assert snapshot( | ||
| process.out, | ||
| path(process.out.versions[0]).yaml | ||
| ).match() } | ||
| ) | ||
| } | ||
|
|
||
| } | ||
|
|
||
| } | ||
78 changes: 78 additions & 0 deletions
78
modules/local/generate_assembly_manifest/tests/main.nf.test.snap
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| { | ||
| "GENERATE_ASSEMBLY_MANIFEST completes with expected outputs - stub": { | ||
| "content": [ | ||
| { | ||
| "0": [ | ||
| [ | ||
| { | ||
| "id": "test" | ||
| }, | ||
| "test.manifest:md5,d41d8cd98f00b204e9800998ecf8427e" | ||
| ] | ||
| ], | ||
| "1": [ | ||
| "versions.yml:md5,188d7f56d019104ee5883d69eede27b4" | ||
| ], | ||
| "manifest": [ | ||
| [ | ||
| { | ||
| "id": "test" | ||
| }, | ||
| "test.manifest:md5,d41d8cd98f00b204e9800998ecf8427e" | ||
| ] | ||
| ], | ||
| "versions": [ | ||
| "versions.yml:md5,188d7f56d019104ee5883d69eede27b4" | ||
| ] | ||
| }, | ||
| { | ||
| "GENERATE_ASSEMBLY_MANIFEST": { | ||
| "python": "Python 3.14.0" | ||
| } | ||
| } | ||
| ], | ||
| "meta": { | ||
| "nf-test": "0.9.3", | ||
| "nextflow": "25.10.0" | ||
| }, | ||
| "timestamp": "2025-10-29T11:00:23.072784" | ||
| }, | ||
| "GENERATE_ASSEMBLY_MANIFEST completes with expected outputs": { | ||
| "content": [ | ||
| { | ||
| "0": [ | ||
| [ | ||
| { | ||
| "id": "test" | ||
| }, | ||
| "233126d4c4d023f18c7836ed36395e3c.manifest:md5,b31fec00db575da44ad17950ae2e37ff" | ||
| ] | ||
| ], | ||
| "1": [ | ||
| "versions.yml:md5,188d7f56d019104ee5883d69eede27b4" | ||
| ], | ||
| "manifest": [ | ||
| [ | ||
| { | ||
| "id": "test" | ||
| }, | ||
| "233126d4c4d023f18c7836ed36395e3c.manifest:md5,b31fec00db575da44ad17950ae2e37ff" | ||
| ] | ||
| ], | ||
| "versions": [ | ||
| "versions.yml:md5,188d7f56d019104ee5883d69eede27b4" | ||
| ] | ||
| }, | ||
| { | ||
| "GENERATE_ASSEMBLY_MANIFEST": { | ||
| "python": "Python 3.14.0" | ||
| } | ||
| } | ||
| ], | ||
| "meta": { | ||
| "nf-test": "0.9.3", | ||
| "nextflow": "25.10.0" | ||
| }, | ||
| "timestamp": "2025-10-29T11:00:16.801535" | ||
| } | ||
| } |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.