Adding configuration file validation for more file formats

[OrangeUtan]

At the start beet only supported Json configuration files. Therefore the extension shipped with built-in validation using Json schemas.

However, since v0.27.0 beet now also supports *.yaml, *.yml, *.toml and pyproject.toml as configuration files. I think it would be usefull to add validation for these files.

Now there is the challlenge of how this will be implemented. Json validation is built-in to Vscode, YAML and toml unfortunately not.

YAML

YAML is a superset of Json and can therefore use the same schemas that are used for Json. The validation itself however requires an extension.
A possible solution would be to use the YAML extension from RedHat, since it supports schemas from SchemaStore. If we add the existing Json schemas to SchemaStore, they can be used for both Json and Yaml validation. Schemas would no longer be packaged inside the extension and could be updated independently.

Toml / pyproject.toml

I'm still searching for a toml validator. Again, relying on an extension is probably the best solution.

[OrangeUtan]

It seems Even Better Toml is an extension that supports toml validation using Json schemas.
But instead of SchemaStore it uses its own remote schema registry.

[vberlier]

Nice suggestion! That seems like a pretty reasonable solution. The YAML extension from RedHat is great, and I didn't know about Even Better Toml but it looks good as well. I've actually been thinking that maybe there's a way to alleviate the maintenance burden of keeping the schema up-to-date by leveraging pydantic to generate json schemas automatically. Currently this would only work for the main config because plugins that retrieve configuration from the context metadata kind of do their own thing. However I'm trying to figure out a way for plugins to use pydantic models to provide runtime validation, and this would potentially make it possible to resolve the schema for plugins statically as well.

[vberlier]

Also I investigated a bit further and it looks like the YAML extension allows other extensions to contribute schemas directly https://github.com/redhat-developer/vscode-yaml#mapping-a-schema-in-an-extension

[OrangeUtan]

Nice find, I should have read the docs more ;).
I will try to implement this solution for Yaml. Maybe Even Better Toml has something similar, if not I can open a feature request.
This will not change our schemas setup for now, so no refactoring neccessary. I think this a good first step and when it works, we can implement generated schemas

[OrangeUtan]

Just confirmed the solution for Yaml works 🎉. I will make a new release right now.

Edit: 0.11.0 now has linting/completions for yaml config files

[OrangeUtan]

Even Better Toml sadly does not have a similar feature, but the maintainer is open to the idea (tamasfe/taplo#130 (comment)) of adding it.

I will try to work in that direction, as it seems to be the most optimal solution.

[vberlier]

Just tried out the yaml autocompletion in the new version it works great! Glad to see that at some point we might be able to do the same for toml. In the meantime I made a first step towards automatically generating schemas by introducing a @configurable plugin decorator that can use pydantic models as validators. Now that I see the improvement it makes I wish I introduced something like this sooner. No more awkward factory functions!