Service discovery bridge for LOOPP: generate Prometheus targets from /discovery

[george-dorin]

Introduce a lightweight Bash SD-merge script that queries each worker node’s /discovery endpoint, merges results, and writes a single file_sd JSON for Prometheus. Each target is labeled with container_name and scrape_path for clean attribution and dashboards.

---

Below is a summarization created by an LLM (gpt-4-0125-preview). Be mindful of hallucinations and verify accuracy.

Why

The changes introduce a new SD-bridge script for querying nodes' /discovery endpoints and generating Prometheus targets. This facilitates the automatic collection of LOOPP metrics, enhancing observability. Additionally, adjustments have been made to the Prometheus configuration to include this new job and support file-based service discovery. The docker-compose file has been updated to include a new service that periodically runs this script, ensuring Prometheus targets are kept up to date without manual intervention.

What

• framework/.changeset/v0.10.18.md
  • Added a changeset document describing the introduction of the SD-bridge script for automatic Prometheus target generation.
• framework/observability/compose/conf/prometheus.yml
  • Added a new job 'node-sd' with file_sd_configs to utilize the generated JSON for service discovery.
• framework/observability/compose/docker-compose.yaml
  • Introduced a new sd-merge service that periodically runs the SD-bridge script, generating a merged JSON file for Prometheus service discovery.
  • Added a new volume sd-targets to store and share service discovery targets between the sd-merge service and Prometheus.
• framework/observability/compose/scripts/sd-merge.sh
  • Added a new script that queries Docker container metadata and their respective /discovery endpoints to generate a comprehensive list of targets for Prometheus, including handling of dynamic IP addresses and custom service discovery labels.