too many buckets in Cloud Run metrics

[WojciechKuk]

I started using Cloud Run and added run.googleapis.com to the prefixes in stackdriver_exporter

I have only ~50 runs/containers, but the exporter started publishing 150,000 metrics and when I looked at them, it turned out that 99% of them were _bucket metrics

for example:

• stackdriver_cloud_run_revision_run_googleapis_com_container_probe_attempt_latencies_bucket geneates 235 buckets per EVERY container (container_probe_attempt_latencies distribution x 235 buckets)

buckets looks like this:

[235 lines]
le="717.9517789085845"
le="789.7469567994431"
le="868.7216524793874"
le="955.5938177273262"
le="1051.153199500059"
le="1156.2685194500648"
le="1271.8953713950714"
le="1399.0849085345785"
le="1538.9933993880366"
le="1692.8927393268405"
le="1862.1820132595244"
le="2048.400214585477"
le="2253.2402360440246"
le="2478.564259648427"
le="2726.420685613271"
le="2999.0627541745976"
le="3298.9690295920577"
le="3628.865932551264"
...

• stackdriver_cloud_run_revision_run_googleapis_com_container_cpu_utilizations_bucket publishes 102 buckets per EVERY container (container_cpu_utilizations distribution x 102 buckets)

buckets looks like this:

le="0.00"
le="0.01"
....[
le="0.94"
le="0.95"
le="0.96"
le="0.97"
le="0.98"
le="0.99"
...

(every percent has it's own bucket)

how to deal with it? is it possible to customize number of retrieved buckets during querying googleapi, or maybe it's something on GCP side (some settings hard to find)?

[SuperQ]

Because Prometheus histograms a are cumulative you can drop them via metric relabel configs.

There is also the option convert_classic_histograms_to_nhcb that can be used to convert these to native histograms.

[WojciechKuk]

yes, I know, that I can drop them, or convert.

more problematic is that [if I understand correctly], these metrics are generated on the Google side and downloading such a large number of metrics simply costs money (google monitoring API requests) and in the future Google billing model (charging for volume instead of number of requests), it will be even more expensive.
maybe the exporter should have an option to ignore (not download at all from the Google API) DISTRIBUTION type metrics (collect only sum and count, skipping buckets for selected metric names), or use some negative regexp to ignore metrics matching a certain pattern?