Skip to content

Commit 941b834

Browse files
committed
Update Prometheus monitoring documentation to include Grafana integration and new dashboards
- Made section less specific to prometheus-community - Added steps to install Grafana - Introduced two new Grafana dashboard JSON files: `environment_overview_dashboard.json` and `freshness_overview_dashboard.json`.
1 parent 586d1da commit 941b834

File tree

5 files changed

+2791
-32
lines changed

5 files changed

+2791
-32
lines changed
Lines changed: 41 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
---
2-
title: "Prometheus Community Helm Chart"
3-
description: "How to monitor the performance and overall health of your Materialize instance using the prometheus-community helm chart."
2+
title: "Grafana using Prometheus"
3+
description: "How to monitor the performance and overall health of your Materialize instance using Prometheus and Grafana."
44
menu:
55
main:
66
parent: "monitor"
77
weight: 1
88
---
99

1010
This guide walks you through the steps required to monitor the performance and
11-
overall health of your Materialize instance using the [`prometheus-community/prometheus`](https://github.com/prometheus-community/helm-charts/tree/main/charts/prometheus) helm chart.
11+
overall health of your Materialize instance using Prometheus and Grafana.
1212

1313
## Before you begin
1414

@@ -22,51 +22,60 @@ Ensure you have:
2222
This guide assumes you have administrative access to your Kubernetes cluster and the necessary permissions to install Prometheus.
2323
{{< /important >}}
2424

25-
## 1. Install Prometheus to your Kubernetes cluster using [`prometheus-community/prometheus`](https://github.com/prometheus-community/helm-charts/tree/main/charts/prometheus)
25+
## 1. Download our Prometheus scrape configurations (`prometheus.yml`)
26+
```bash
27+
curl -O https://raw.githubusercontent.com/MaterializeInc/materialize/refs/heads/self-managed-docs/v25.2/doc/user/data/monitoring/prometheus.yml
28+
```
2629

27-
1. Download the Materialize Prometheus scrape configuration file:
28-
```bash
29-
curl -O https://raw.githubusercontent.com/MaterializeInc/materialize/refs/heads/self-managed-docs/v25.2/doc/user/data/monitoring/prometheus.yml
30-
```
3130

32-
2. Download the prometheus-community default chart values:
31+
## 2. Install Prometheus to your Kubernetes cluster using [`prometheus-community/prometheus`](https://github.com/prometheus-community/helm-charts/tree/main/charts/prometheus) (Optional)
32+
33+
1. Download the prometheus-community default chart values (`values.yaml`):
3334
```bash
3435
curl -O https://raw.githubusercontent.com/prometheus-community/helm-charts/refs/heads/main/charts/prometheus/values.yaml
3536
```
3637

37-
3. Replace `values.yaml`'s `serverFiles > prometheus.yml > scrape_configs` with `prometheus.yml`'s `scrape_configs`. It should look something like:
38-
```yml
39-
serverFiles:
40-
prometheus.yml:
41-
scrape_configs:
42-
- job_name: kubernetes-pods
43-
...
44-
```
45-
3. Create a `prometheus` namespace
38+
2. Within `values.yaml`, replace `serverFiles > prometheus.yml > scrape_configs` with our scrape configurations (`prometheus.yml`).
39+
40+
3. Install the operator with the updated `values.yaml`
4641
```bash
4742
kubectl create namespace prometheus
48-
```
49-
4. Install the operator with the scrape configuration
50-
```bash
5143
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
5244
helm repo update
5345
helm install --namespace prometheus prometheus prometheus-community/prometheus \
5446
--values values.yaml
5547
```
5648

49+
## 3. Visualize through Grafana (optional)
5750

58-
## 2. Validate through the Prometheus UI
51+
1. Install the Grafana helm chart following [this guide](https://grafana.com/docs/grafana/latest/setup-grafana/installation/helm/)
5952

60-
{{< note >}}
61-
The port forwarding method described below is for testing purposes only. For production environments, configure an ingress controller to securely expose the Prometheus UI.
62-
{{< /note >}}
6353

64-
1. Set up port forwarding to access the Prometheus UI:
54+
2. Set up port forwarding to access the Grafana UI:
55+
```bash
56+
MZ_POD_GRAFANA=$(kubectl get pods -n monitoring -l app.kubernetes.io/name=grafana -o custom-columns="NAME:.metadata.name" --no-headers)
57+
kubectl port-forward pod/$MZ_POD_GRAFANA 3000:3000 -n monitoring
58+
```
6559

66-
```bash
67-
MZ_POD_PROMETHEUS=$(kubectl get pods -n prometheus -l app.kubernetes.io/name=prometheus -o custom-columns="NAME:.metadata.name" --no-headers)
68-
kubectl port-forward pod/$MZ_POD_PROMETHEUS 9090:9090 -n prometheus
69-
```
60+
{{< note >}}
61+
The port forwarding method is for testing purposes only. For production environments, configure an ingress controller to securely expose the Grafana UI.
62+
{{< /note >}}
63+
64+
3. Within the UI, add a Prometheus data source where the URL is `http://<prometheus server name>.<namespace>.svc.cluster.local:<port>`(i.e. `http://prometheus-server.prometheus.svc.cluster.local:80`)
65+
66+
![Image of Materialize Console login screen with mz_system user](/images/grafana-prometheus-datasource-setup.png)
67+
68+
4. Download the following dashboards:
69+
```bash
70+
# Environment overview: An overview of the state of different objects in your environment
71+
curl -O https://raw.githubusercontent.com/MaterializeInc/materialize/refs/heads/self-managed-docs/v25.2/doc/user/data/monitoring/grafana_dashboards/environment_overview_dashboard.json
72+
```
73+
74+
```bash
75+
# Freshness overview: An overview of how out of date objects in your environment are
76+
curl -O https://raw.githubusercontent.com/MaterializeInc/materialize/refs/heads/self-managed-docs/v25.2/doc/user/data/monitoring/grafana_dashboards/freshness_overview_dashboard.json
77+
```
78+
79+
5. [Import the dashboards using the Prometheus data source](https://grafana.com/docs/grafana/latest/dashboards/build-dashboards/import-dashboards/#importing-a-dashboard)
7080

71-
2. Access the Prometheus UI by navigating to `localhost:9090` in your web
72-
browser.
81+
![Image of Materialize Console login screen with mz_system user](/images/grafana-monitoring-success.png)

0 commit comments

Comments
 (0)