Skip to content

Implement nf-schema in pipeline #472

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 5 commits into
base: dev
Choose a base branch
from
Draft

Implement nf-schema in pipeline #472

wants to merge 5 commits into from

Conversation

edmundmiller
Copy link

The pipeline's sample sheet validation was fully migrated to nf-schema, replacing a custom Python script.

Key changes include:

  • assets/schema_input.json was updated to:
    • Implement comprehensive regex patterns for sample, antibody, and control fields.
    • Enforce antibody/control dependencies using dependentRequired.
    • Add minimum: 1 constraint for replicate IDs.
  • The custom Python validation script bin/check_samplesheet.py and its Nextflow wrapper modules/local/samplesheet_check.nf were removed.
  • subworkflows/local/utils_nfcore_chipseq_pipeline/main.nf gained a validateSamplesheetRow() function to preserve pre-nf-schema logic, including space replacement warnings and single_end computation.
  • subworkflows/local/input_check.nf was simplified, removing its dependency on the Python script and now directly processing pre-validated samplesheet data.
  • main.nf and workflows/chipseq.nf were updated to use the PIPELINE_INITIALISATION.out.samplesheet channel, reflecting the new validation flow.

Subsequently, only nf-test related artifacts were removed, including nextflow and nf-test executables, and all .nf-test/ directories, while preserving all nf-schema implementation changes.

@nf-core-bot
Copy link
Member

Warning

Newer version of the nf-core template is available.

Your pipeline is using an old version of the nf-core template: 3.2.1.
Please update your pipeline to the latest version.

For more documentation on how to update your pipeline, please see the nf-core documentation and Synchronisation documentation.

@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 1, 2025 •
edited
Loading

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 96867d9

+| ✅ 236 tests passed       |+
#| ❔   2 tests were ignored |#
!| ❗  32 tests had warnings |!

❗ Test warnings:

  • pipeline_todos - TODO string in ro-crate-metadata.json: "description": "

    \n \n <source media="(prefers-color-scheme: dark)" srcset="docs/images/nf-core-chipseq_logo_dark.png">\n <img alt="nf-core/chipseq" src="docs/images/nf-core-chipseq_logo_light.png">\n \n

    \n\nGitHub Actions CI Status\nGitHub Actions Linting StatusAWS CICite with Zenodo\nnf-test\n\nNextflow\nrun with conda\nrun with docker\nrun with singularity\nLaunch on Seqera Platform\n\nGet help on SlackFollow on TwitterFollow on MastodonWatch on YouTube\n\n## Introduction\n\nnf-core/chipseq is a bioinformatics pipeline that ...\n\n TODO nf-core:\n Complete this sentence with a 2-3 sentence summary of what types of data the pipeline ingests, a brief overview of the\n major pipeline sections and the types of output it produces. You're giving an overview to someone new\n to nf-core here, in 15-20 seconds. For an example, see https://github.com/nf-core/rnaseq/blob/master/README.md#introduction\n\n\n Include a figure that guides the user through the major workflow steps. Many nf-core\n workflows use the "tube map" design for that. See https://nf-co.re/docs/contributing/design_guidelines#examples for examples. \n Fill in short bullet-pointed list of the default steps in the pipeline 1. Read QC (FastQC)2. Present QC for raw reads (MultiQC)\n\n## Usage\n\n> [!NOTE]\n> If you are new to Nextflow and nf-core, please refer to this page on how to set-up Nextflow. Make sure to test your setup with -profile test before running the workflow on actual data.\n\n Describe the minimum required steps to execute the pipeline, e.g. how to prepare samplesheets.\n Explain what rows and columns represent. For instance (please edit as appropriate):\n\nFirst, prepare a samplesheet with your input data that looks as follows:\n\nsamplesheet.csv:\n\ncsv\nsample,fastq_1,fastq_2\nCONTROL_REP1,AEG588A1_S1_L002_R1_001.fastq.gz,AEG588A1_S1_L002_R2_001.fastq.gz\n\n\nEach row represents a fastq file (single-end) or a pair of fastq files (paired end).\n\n\n\nNow, you can run the pipeline using:\n\n update the following command to include all required parameters for a minimal example \n\nbash\nnextflow run nf-core/chipseq \\\n -profile <docker/singularity/.../institute> \\\n --input samplesheet.csv \\\n --outdir <OUTDIR>\n\n\n> [!WARNING]\n> Please provide pipeline parameters via the CLI or Nextflow -params-file option. Custom config files including those provided by the -c Nextflow option can be used to provide any configuration except for parameters; see docs.\n\nFor more details and further functionality, please refer to the usage documentation and the parameter documentation.\n\n## Pipeline output\n\nTo see the results of an example test run with a full size dataset refer to the results tab on the nf-core website pipeline page.\nFor more details about the output files and reports, please refer to the\noutput documentation.\n\n## Credits\n\nnf-core/chipseq was originally written by Espinosa-Carrasco J, Patel H, Wang C, Ewels P.\n\nWe thank the following people for their extensive assistance in the development of this pipeline:\n\n If applicable, make list of people who have also contributed \n\n## Contributions and Support\n\nIf you would like to contribute to this pipeline, please see the contributing guidelines.\n\nFor further information or help, don't hesitate to get in touch on the Slack #chipseq channel (you can join with this invite).\n\n## Citations\n\n Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file. \n If you use nf-core/chipseq for your analysis, please cite it using the following doi: 10.5281/zenodo.XXXXXX \n\n Add bibliography of tools and data used in your pipeline \n\nAn extensive list of references for the tools used by the pipeline can be found in the CITATIONS.md file.\n\nYou can cite the nf-core publication as follows:\n\n> The nf-core framework for community-curated bioinformatics pipelines.\n>\n> Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.\n>\n> Nat Biotechnol. 2020 Feb 13. doi: 10.1038/s41587-020-0439-x.\n",
  • pipeline_todos - TODO string in nextflow.config: Optionally, you can add a pipeline-specific nf-core config at https://github.com/nf-core/configs
  • pipeline_todos - TODO string in nextflow.config: Update the field with the details of the contributors to your pipeline. New with Nextflow version 24.10.0
  • pipeline_todos - TODO string in base.config: Check the defaults for all processes
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
  • pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
  • pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
  • pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
  • local_component_structure - igv.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - star_align.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - gtf2bed.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - macs3_consensus.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - bam_remove_orphans.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - plot_homer_annotatepeaks.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - genome_blacklist_regions.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - multiqc.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - plot_macs3_qc.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - samplesheet_check.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - multiqc_custom_phantompeakqualtools.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - bamtools_filter.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - multiqc_custom_peaks.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - deseq2_qc.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - star_genomegenerate.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - annotate_boolean_peaks.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - frip_score.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - bam_peaks_call_qc_annotate_macs3_homer.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - bed_consensus_quantify_qc_bedtools_featurecounts_deseq2.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - bam_filter_bamtools.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - prepare_genome.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - bam_bedgraph_bigwig_bedtools_ucsc.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - align_star.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - input_check.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure

❔ Tests ignored:

  • nextflow_config - Config default ignored: params.bamtools_filter_pe_config
  • nextflow_config - Config default ignored: params.bamtools_filter_se_config

✅ Tests passed:

Run details

  • nf-core/tools version 3.2.1
  • Run at 2025-09-03 15:16:22

cursoragent and others added 5 commits September 3, 2025 08:33
@cursoragent @edmundmiller
fix: Use --bga in bedtools genomecov for more comprehensive coverage …
5bc2b9e 
…output
@edmundmiller @pinin4fjords
fix: Move bg to args
d51ec4b 
#470 (comment)

Co-authored-by: pinin4fjords <[email protected]>
@cursoragent @edmundmiller
Change bigWig generation to use -bga for better IGV visualization
07c95f8
@edmundmiller
chore: nf-core modules install bedtools/genomecov
f67b5ad
@edmundmiller @claude
refactor: Replace local bedtools genomecov module with nf-core version
96867d9 
- Replace local bedtools_genomecov module with nf-core bedtools/genomecov
- Add inline scale factor calculation from flagstat in subworkflow
- Configure nf-core module with -bga and -pc flags for proper output
- Remove scale_factor output emit as it's no longer needed
- Maintain same functionality while using official nf-core modules

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@edmundmiller edmundmiller force-pushed the cursor/implement-nf-schema-in-pipeline-14ac branch from 29520d7 to 96867d9 Compare September 3, 2025 15:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@edmundmiller @nf-core-bot @cursoragent