[Associated vote ended on 2023-03-09] Include semantic markup in plugin DOCUMENTATION strings

[samccann]

Summary

Community Ansible documentation wants to add semantic markup within the module/plugin DOCUMENTATION strings. This gives us improved flexibility on how we display this information.

Additional Information

Background
This started with a general dissatisfaction on how some of the module options and parameters are displayed on docs.ansible.com (what is bold, what is italics, monospace, etc) and the inherent inflexibility of our current markup options.

Primary goal - To improve the visual aid that helps readers determine what the differences.​
Secondary goal - Do this in such a way that we do not have to touch the code in ansible/ansible again if we decide we need to display this information in a different way.
Solution - Introduce a light level of semantic markup, where the developer 'marks' the content for what it is (module, value, option, etc) instead of how it should display (bold, italics, monospace, etc). This leaves the visual aspect to separate code we can update at will.

Specific changes
The community decided on the following additions:

• O(key) and O(key=value) for option names, with or without values
• V(value) for standalone option values
• E(env_var) for environment variables
• P(FQCN#plugintype) for plugin references

Existing markup macros will remain, including:

• M(FQCN) for module references
• C(code) for other monospace text such as linux commands
• B(bold_text) and I(italic_text) for other words or phrases an author wants to emphasize

These added macros may impact other projects, such as Galaxy-ng/Automation Hub, ansible-navigator, possibly the vscode plugin, and the collection_prep script used by some collections to generate local rst docs files.

For more complete details and WIP PRs, see ansible/ansible#75054.

[felixfontein]

Here's a proposal to extend O(), to allow it to a) validate its contents (make sure you reference options that actually exist!) and b) convert them to links (so users can click on them and end up at the documentation of the option that's mentioned).

The following forms of O(...) are allowed:

1. O(name) or O(name=value) for a top-level option name of the current plugin/module;
2. O(name.sub) or O(name.sub=value) for a suboption sub of the top-levle option name of the current plugin/module; if you have sub-sub-options, use O(name.sub.subsub) etc;
3. O(FQCN#type:name) or O(FQCN#type:name=value) for a top-level option name of the plugin/module FQCN (f.ex. ansible.builtin.uri) of plugin type type (one of module, lookup, ..., role);
4. O(FQCN#type:name.sub.subsub) or O(FQCN#type:name.sub.subsub=value) for a sub-sub-option subsub of a sub-option sub of a top-level option name of the plugin/module FQCN (f.ex. ansible.builtin.uri) of plugin type type (one of module, lookup, ..., role).

This looks complicated, but as opposed to what we currently have, and what's currently proposed, this allows validation and allows to create links. Retrospectively adding validation and links is not really possible if we allow the free-form version of O(...).

[felixfontein]

With that I'd also like to propose a markup for return values. Similar as O(...) for options, this can be used for stricter validation and linking. Basically I suggest to use RV(name) or RV(name=value), with the same adjustments for sub-return values and return values of other modules/plugins as for O(...) above. So the generic form will be RV(FQCN#type:name.sub.subsub=value). (For facts, one can simply use RV(FQCN#type:ansible_facts.name.sub.subsub).)

[felixfontein]

It probably makes sense to allow to leave away FQCN and plugin type in O() and R() if they refer to the current plugin/module. This still allows to do proper linking and proper validation, and covers the huge majorities of usages of these markups.

[samccann]

Included the above details into a specification at https://hackmd.io/VjN60QSoRSSeRfvGmOH1lQ

We can use that hackmd doc to finalize our proposed specification and then get it approved by the other projects that depend on this ansible-doc --json output.

[felixfontein]

Another improvement I'd like to suggest: Allow to escape ) and \ by using \ in values, i.e. in the parts after = for O() and RV(), and in the whole of V(). Then it will be possible to include ), which right now is not possible with C(), I() and B().

(I would not allow this for C(), I() and B(), since this would be a breaking change in case someone is using something like C(\), which right now is a valid markup for generating \, but with such a change would generate ) followed by more code-style content until the next ) in the input.)

(I already added it to https://hackmd.io/VjN60QSoRSSeRfvGmOH1lQ. Also please note that this makes parsing a bit more complicated, but one can still use regular expressions to find the markup parts.)

[felixfontein]

I've continued on the implementation side of this:

• [WIP] Add new semantic markup support to antsibull-docs antsibull-build#281 now supports all forms of O() and RV();
• Use semantic markup ansible-collections/community.dns#23 uses this;
• Add new semantic markup support to ansible-doc text output and to validate-modules ansible/ansible#74937 supports this markup in ansible-doc output, and adds ansible-test validate-modules support both for generic syntax, and for making sure that local option names and return value names mentioned by semantic markup actually exist.

This includes another extension: allowing to use [] in option and return value names; see ansible-community/antsibull-build#281 (comment) for details. (You can use it for things like records[].value to reference the suboption value inside the list records.)