SPNG processing campaigns. #25
Description
At the hackathon we came up with a list of four initial processing campaigns intended to help push us to production scale.
This issue is intended to be the parent of an issue for each campaign.
The four campaigns are summarized and see below for links to per-campaign issue with details.
- Shake down campaign : Goal: run something with SPNG to shake out problems in code, configuration and infrastructure the development has yet to catch. Sim only and output need not necessarily be useful other than it should demonstrate no glaring bugs. Include CPU-only + CPU+GPU modes.
- SPDIR campaign : Goal: run OSP, SPNG and Splat to produce metrics for producing "SPDIR" plots (efficiency/bias/resolution). May not necessarily need batch system but could be used to test out a DAG of jobs.
- Full-chain campaign: Goal, run both OSP and SPNG in art/larsoft as part of a full chain of simulation and signal processing using PDHD cosmic muons. Include processing to compare OSP and SPNG output to produce files of metrics or even also final plots.
- 1M event campaign: Goal, scale up to the level of days of FD data. Use PDHD simulation and PDHD data, include OSP and SPNG and metric evaluation as in full-chain. May be run by the prompt-processing system itself. This campaign will produce a lot of data useful to a lot of collaborators and should be executed with close coordination with DUNE production team.