[RFC, DO NOT MERGE] Spec for generating machine-parseable .md files for AI agents

content/en/md_conversion_test_files/README.md

@@ -0,0 +1,24 @@
+---
+title: Plaintext Markdown Spec
+---
+
+## What is this?
+
+This folder contains a listing of the components used on the docs site, and a description of their desired output format in the plaintext files generated for AI agents.
+
+The HTML compiled from these files will be used as the test input HTML for the Markdown generator job.
+
+## Where can I learn more?
+
+1. Read [Datadog Docs for AI Agents](https://docs.google.com/document/d/1t48N0uDJElMP_228n2bIYi_cUIj_xMOKjdHuh1Z4rMg), which attempts to summarize the problem space and why this is important.
+2. Ask Jen Gilbert.
+
+## Disclaimer
+
+Everything described here is *probably* possible based on the technical context and existing libraries we have so far, but it's not *guaranteed* to be possible until the docs are migrated to Markdoc.
+
+If you'd like to follow along with changes in the spec, Jen will keep this PR up to date, with clear commit messages about what changed.
+
+## Feedback is welcome!
+
+Jen took her best shot at these, but she's not attached to any of it. Comment away!

content/en/md_conversion_test_files/common_components/alert.md

@@ -0,0 +1,25 @@
+---
+title: Alert
+---
+
+## Expected .md output
+
+Each alert is enclosed in `{% alert %}` and `{% /alert %}` tags, with a `level` attribute.
+
+### Info
+
+<div class="alert alert-info">This is an info alert.</div>
+
+This is a random sentence.
+
+### Warning
+
+<div class="alert alert-warning">This is a warning alert containing an <a href="https://google.com">HTML link.</a></div>
+
+This is a random sentence.
+
+### Danger
+
+<div class="alert alert-danger">This is a danger alert.</div>
+
+This is a random sentence.

content/en/md_conversion_test_files/common_components/callouts.md

@@ -0,0 +1,24 @@
+---
+title: Callouts
+---
+
+## Expected .md output
+
+- Each callout is enclosed in `{% callout %}` and `{% /callout %}` tags.
+- Any attributes, such as the title or the URL, are also included in the box contents:
+    - The title is a heading at the top of the box.
+    - The URL is included with verbiage at the bottom like "Visit this URL: <URL>".
+
+## Example inputs
+
+### Callout (preview / beta)
+
+{{< callout url="https://www.datadoghq.com/">}}
+  Datadog Apps are in Preview. Request access by using this form. Once approved, you can start getting creative and develop your App for you, your organization, or for publishing to the entire Datadog community alongside our other great Datadog Apps!
+{{< /callout >}} 
+
+### Learning center callout
+
+{{< learning-center-callout header="Try Foundations in the Learning Center" btn_title="Enroll Now" btn_url="https://learn.datadoghq.com/courses/datadog-foundation">}}
+  The Datadog Learning Center is full of hands-on courses to help you learn about this topic. Enroll today to learn more about Datadog Foundations.
+{{< /learning-center-callout >}}

content/en/md_conversion_test_files/common_components/code_blocks.md

@@ -0,0 +1,60 @@
+---
+title: Code Blocks
+---
+
+## Scope
+
+This covers the `code-block` and `highlight` shortcodes. Plain fences are already covered in the "plain markdown" examples.
+
+## Expected .md output
+
+Each code block is wrapped in a fence, with the language included when available:
+
+```lang
+Code contents here.
+```
+
+### Filenames
+
+When a filename is present, it appears above the code block, like this:
+
+In the `block.java` file:
+
+```lang
+Code contents here.
+```
+
+### Highlighted lines
+
+When a highlight is present, it appears above the code block, like this:
+
+See lines 2-3 below:
+
+```lang
+Code contents here.
+```
+
+## Example inputs
+
+### Code block
+
+{{< code-block lang="java" filename="block.java" disable_copy="true" collapsible="true" >}}
+import com.datadoge.docs.SweetCodeBlock;
+import com.datadoge.docs.TryItOut;
+public class CodeBlocksForLife {
+    ...
+    }
+{{< /code-block >}}
+
+### Highlighted code block
+
+{{< highlight yaml "hl_lines=6-8" >}}
+    apiVersion: apps/v1
+    kind: Deployment
+    metadata:
+    name: springfront
+    labels:
+        tags.datadoghq.com/env: "dev"
+        tags.datadoghq.com/service: "springfront"
+        tags.datadoghq.com/version: "12"
+    {{< /highlight >}}

content/en/md_conversion_test_files/common_components/collapsible_sections.md

@@ -0,0 +1,30 @@
+---
+title: Collapsible Sections
+---
+
+## Expected .md output
+
+- Each section is enclosed in `{% collapsible-section %}` and `{% /collapsible-section %}` tags. Any attributes, like `title`, are included in the tag.
+
+## Example inputs
+
+### Details
+
+This content is outside the `details` tag.
+
+<details>
+  <summary>Details</summary>
+  This content would be hidden inside the details panel.
+</details>
+
+This content is outside the `details` tag.
+
+### Collapse content
+
+This content is outside the `content-collapse` block.
+
+{{% collapse-content title="Datadog Operator" level="h4" expanded=false id="id-for-anchoring" %}}
+This content is inside the `content-collapse` block.
+{{% /collapse-content %}} 
+
+This content is outside the `content-collapse` block.

content/en/md_conversion_test_files/common_components/further_reading.md

@@ -0,0 +1,20 @@
+---
+title: Further Reading
+further_reading:
+  - link: "logs/processing/pipelines"
+    tag: "Documentation"
+    text: "Log processing pipelines"
+  - link: "some/blog/post"
+    tag: "Blog"
+    text: "Some blog post"
+---
+
+## Expected .md output
+
+A list of links, introduced by the text "Further reading:".
+
+## Example input
+
+## Further reading
+
+{{< partial name="whats-next/whats-next.html" >}}

content/en/md_conversion_test_files/common_components/images_and_videos.md

@@ -0,0 +1,17 @@
+---
+title: Images and Videos
+---
+
+## Expected .md output
+
+An easily readable tag is used for each one, such as `{% image src="path/to/image" %}`, with any available attributes.
+
+## Example inputs
+
+## Image
+
+{{< img src="path/to/your/image-name-here.png" alt="Your image description" caption="Your image description" style="width:100%;" >}}
+
+## Video
+
+{{< img src="path/to/your/video-name-here.mp4" alt="Your image description" video="true" width="100%" >}}

content/en/md_conversion_test_files/common_components/plain_markdown.md

@@ -0,0 +1,80 @@
+---
+title: Plain Markdown Elements
+---
+
+## Expected .md output
+
+- Most of this should remains unchanged, or uses a format that does not meaningfully differ from its original form. (Some spacing changes can be difficult to avoid.)
+- In some cases, content may be slightly optimized for agents (for example, links will become inline).
+
+## Headings
+
+### Heading level 3
+
+Some text.
+
+#### Heading level 4 {#heading-level-4}
+
+Some text, with a custom heading ID.
+
+##### Heading level 5
+
+Some text.
+
+###### Heading level 6
+
+Some text.
+
+## Inline text formatting
+
+You can *italicize* text, or **bold** it.
+
+Text can also be formatted as a [plain link][1], **[bold link][1]**, or *[italic link][1]*.
+
+## Lists
+
+### Ordered
+
+Here's an ordered list of steps:
+
+1. Figure out how to migrate to [Astro][2].
+2. Make some *incredible docs*.
+3. **Profit.**
+
+### Unordered
+
+Here's an unordered version of the same list, which doesn't make as much sense, but that's okay.
+
+- Figure out how to migrate to [Astro][2].
+- Make some *incredible docs*.
+- **Profit.**
+
+## Code
+
+### Inline code
+
+Here's some `inline code`.
+
+### Code fence
+
+```javascript
+console.log("Hello, world!");
+```
+
+## Description list
+
+Service
+: Services are the building blocks of modern microservice architectures - broadly a service groups together endpoints, queries, or jobs for the purposes of building your application.
+
+Resource
+: Resources represent a particular domain of a customer application - they are typically an instrumented web endpoint, database query, or background job.
+
+`clusterChecksRunner.affinity.podAffinity.preferredDuringSchedulingIgnoredDuringExecution`
+: Required. A list of node selector terms. The terms are ORed.
+
+`site`
+: Set the site of the Datadog intake for Agent data:  {{< region-param key="dd_site" code="true" >}}. Defaults to `datadoghq.com`.
+
+
+[1]: https://www.google.com
+[2]: https://astro.build/

content/en/md_conversion_test_files/common_components/programming_lang_wrapper.md

@@ -0,0 +1,36 @@
+---
+title: Programming lang wrapper
+---
+
+## Expected .md output
+
+These are treated as tabs (see `tabs.md`).
+
+## Example input
+
+{{< programming-lang-wrapper langs="python,ruby,PHP" >}}
+
+{{< programming-lang lang="python" >}}
+Info about how to do this with Python.
+ ```python
+from datadog import initialize, statsd
+```
+{{< /programming-lang >}}
+
+{{< programming-lang lang="ruby" >}}
+Info about how to do this with Ruby.
+ ```ruby
+require 'datadog/statsd'
+```
+{{< /programming-lang >}}
+
+{{< programming-lang lang="PHP" >}}
+Info about how to do this with PHP.
+ ```php
+<?php
+
+require __DIR__ . '/vendor/autoload.php';
+```
+{{< /programming-lang >}}
+
+{{< /programming-lang-wrapper >}}

content/en/md_conversion_test_files/common_components/region_shortcodes.md

@@ -0,0 +1,36 @@
+---
+title: Region Shortcodes
+---
+
+## Site region shortcode
+
+### Expected .md output
+
+The contents are enclosed in `[BEGIN CONTENT FOR REGIONS "ap1", "gov"]` and `[END CONTENT FOR REGIONS "ap1", "gov"]`.
+
+### Example input
+
+{{< site-region region="us,ap1" >}}
+This paragraph only appears if the user has selected US or AP1 from the Site Selector dropdown.
+{{< /site-region >}}
+
+## Region parameter shortcode
+
+### Expected .md output
+
+A placeholder: 
+
+- `<YOUR_DATADOG_SITE>` in code.
+- `[USER'S DATADOG SITE]` in copy.
+
+### Example inputs
+
+#### In code
+
+```shell
+export DD_SITE={{< region-param key=dd_site code="true" >}}
+```
+
+#### In copy
+
+Your Datadog site is {{< region-param key=dd_site >}}.