Conditional metadata validation or multiple schema?

[mhpob]

I've hit a bump in terms of metadata validation while trying to add some MATOS data submission forms: these will have an inherently different metadata structure than, say, receiver detections.

To address this, would it be better to have separate metadata schema for each sub-directory of the repository, or to implement a large conditional schema (if type == detections, validate this way, etc.)?

I feel the former may be less confusing if we outline specifically what is going on. This would require shifting files around in currently-pending PRs (#10 and #11) and adjustment of the validator GH Action, so I'll wait until those are accepted/rejected/edited.

The latter doesn't seem to be possible when using Yamale, but may be possible using whip, if I'm reading this right: https://github.com/inbo/whip/blob/master/docs/syntax.md#if

[jdpye]

Definitely something whip can do. I'm imagining we could instead include the whip spec in the folder it's meant to be applied to if we wanted to be modular about it, but the idea of a centralizing schema would be easier to maintain. let me ruminate some more on this

[jdpye]

whip's philosophy is closer to being able to answer the question: 'this data file has what you expect in order to accomplish a task' vs. 'this metadata is complete', which is what we actually want here.

We would want to evaluate the YAML for completeness, both at the 'you must have this much metadata to join this repository' level and the 'the internal description of the file is complete' level. That's probably best done in yamale and not whip.

[mhpob]

Sequential validation of multiple schema:

1. Should this be in the repo?
2. Should this be in the directory?
3. Is the metadata complete? "leaf level"

[mhpob]

Yamale will recursively validate all files when provided with a directory. 76470f4 moves us toward that -- we can just have a separate step for each level. I imagine the "should this be in the repo" will be the first step to fail quickly, and work down from there.