Replace Nextflow components with Metaflow workflow orchestration

[Copilot]

Summary

This PR completely replaces the Nextflow-based workflow orchestration with Netflix's Metaflow while preserving the entire Python package (af3_linear_epitopes) intact. The migration provides a unified, parameter-driven workflow system that maintains all existing functionality.

Changes Made

🔄 Core Workflow Migration

• Before: 12 separate Nextflow files (.nf) for different datasets and processing stages
• After: Single unified Metaflow workflow (workflows/linear_epitope_flow.py) with parameter-based routing

📋 Workflow Types Supported

• clean: Data cleaning and preparation for all datasets (bp3c50id, hv_class, hv_seg, iedb_bp3, in_class, in_seg)
• msa_focal: MSA generation for focal proteins
• msa_peptide: MSA generation for peptides
• inference_focal: AlphaFold3 inference for focal proteins
• inference_peptide: AlphaFold3 inference for peptides
• bepipred: BepiPred scoring with result joining
• extract_conf: Confidence extraction from AF3 results

🚀 Execution Scripts

• Before: scripts/ with nextflow run commands
• After: scripts_metaflow/ with python linear_epitope_flow.py run commands
• Preserved all SLURM job configurations and resource requirements
• Updated log directories from tmp/nextflow/ to tmp/metaflow/

🔧 Environment Updates

• Added Metaflow dependency to envs/env.yaml
• Deprecated nf-core.yaml (no longer needed)
• Updated .gitignore to exclude Metaflow metadata

Usage Examples

Command Line

cd workflows
python linear_epitope_flow.py run --dset-name bp3c50id --workflow-type clean
python linear_epitope_flow.py run --dset-name hv_class --workflow-type msa_focal

SLURM Submission

sbatch scripts_metaflow/bp3c50id/00_run_clean_raw_data.sh
sbatch scripts_metaflow/bp3c50id/02_run_msa_focal_protein.sh

Key Benefits

✅ Simplified Architecture: One workflow file instead of 12 separate files
✅ Parameter-Driven: Easy switching between datasets and workflow types
✅ Maintained Functionality: All original Nextflow processes preserved
✅ Better Error Handling: Improved command execution and logging
✅ Python Integration: Native Python workflow orchestration

Python Package Preservation

The af3_linear_epitopes Python package in src/ remains completely unchanged:

• All utility functions and modules preserved
• Binary scripts in workflows/bin/ still used by Metaflow workflows
• No breaking changes to existing Python code

Legacy Support

Old Nextflow files are marked as deprecated with clear migration documentation but remain available for reference.

Fixes #30.

---

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.