fix(litellm): [MLOS-182] select metadata keys to tag from litellm kwargs #14067

ncybul · 2025-07-18T15:25:41Z

We received a customer ticket explaining that vertex_credentials was being collected as a metadata field on LLM spans emitted by the LiteLLM integration. This PR addresses that issue and safeguards against potentially other sensitive information being exposed by explicitly selecting a subset of kwargs to attach to the LLM span's metadata. Keys were chosen based on the arguments passed into the LiteLLM SDK completion method and text_completion method.

I verified that this PR does resolve the issue. The following script was run to produce this trace where the metadata field does not contain vertex_credentials.

from litellm import completion
import json 

file_path = '/path/to/credentials'

with open(file_path, 'r') as file:
    vertex_credentials = json.load(file)

vertex_credentials_json = json.dumps(vertex_credentials)

response = completion(
  model="vertex_ai/gemini-1.5-flash",
  messages=[{ "content": "Hello, how are you?","role": "user"}],
  vertex_credentials=vertex_credentials_json,
  stream=True
)
for chunk in response:
    print(chunk)

Checklist

PR author has checked that all the criteria below are met
The PR description includes an overview of the change
The PR description articulates the motivation for the change
The change includes tests OR the PR description describes a testing strategy
The PR description notes risks associated with the change, if any
Newly-added code is easy to change
The change follows the library release note guidelines
The change includes or references documentation updates if necessary
Backport labels are set (if applicable)

Reviewer Checklist

Reviewer has checked that all the criteria below are met
Title is accurate
All changes are related to the pull request's stated goal
Avoids breaking API changes
Testing strategy adequately addresses listed risks
Newly-added code is easy to change
Release note makes sense to a user of the library
If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
Backport labels are set in a manner that is consistent with the release branch maintenance policy

github-actions · 2025-07-18T15:26:11Z

CODEOWNERS have been resolved as:

releasenotes/notes/litellm-scrub-metadata-135109a6a5324111.yaml         @DataDog/apm-python
ddtrace/llmobs/_integrations/litellm.py                                 @DataDog/ml-observability
ddtrace/llmobs/_integrations/utils.py                                   @DataDog/ml-observability

github-actions · 2025-07-18T15:48:31Z

Bootstrap import analysis

Comparison of import times between this PR and base.

Summary

The average import time from this PR is: 287 ± 5 ms.

The average import time from base is: 291 ± 5 ms.

The import time difference between this PR and base is: -4.2 ± 0.2 ms.

Import time breakdown

The following import paths have shrunk:

ddtrace.auto 2.188 ms (0.76%)

ddtrace.bootstrap.sitecustomize 1.500 ms (0.52%)

ddtrace.bootstrap.preload 1.500 ms (0.52%)

ddtrace.internal.remoteconfig.client 0.690 ms (0.24%)

ddtrace 0.688 ms (0.24%)

ddtrace.internal._unpatched 0.034 ms (0.01%)

json 0.034 ms (0.01%)

json.decoder 0.034 ms (0.01%)

re 0.034 ms (0.01%)

enum 0.034 ms (0.01%)

types 0.034 ms (0.01%)

pr-commenter · 2025-07-18T16:25:45Z

Performance SLOs

Benchmark execution time: 2025-07-21 15:23:06

Comparing candidate commit 4986965 in branch nicole-cybul/redact-litellm-params with performance thresholds.

coreapiscenario-context_with_data_listeners

(unstable) execution_time seen in benchmark [13.115µs; 14.217µs]; SLO is < 20.000µs
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

coreapiscenario-context_with_data_no_listeners

🟩 execution_time seen in benchmark [3.684µs; 3.713µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

coreapiscenario-context_with_data_only_all_listeners

(unstable) execution_time seen in benchmark [13.114µs; 14.210µs]; SLO is < 20.000µs
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

coreapiscenario-get_item_exists

🟩 execution_time seen in benchmark [632.184ns; 638.284ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.044MB; 29.112MB]; SLO is < 31.000MB

coreapiscenario-get_item_missing

🟩 execution_time seen in benchmark [684.020ns; 690.730ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

coreapiscenario-set_item

🟩 execution_time seen in benchmark [24.385µs; 24.629µs]; SLO is < 30.000µs
🟩 max_rss_usage seen in benchmark [29.044MB; 29.112MB]; SLO is < 31.000MB

djangosimple-appsec

🟩 execution_time seen in benchmark [21.235ms; 21.294ms]; SLO is < 22.300ms
🟩 max_rss_usage seen in benchmark [63.694MB; 63.787MB]; SLO is < 65.500MB

djangosimple-exception-replay-enabled

🟩 execution_time seen in benchmark [1.355ms; 1.359ms]; SLO is < 1.450ms
🟩 max_rss_usage seen in benchmark [63.407MB; 63.562MB]; SLO is < 65.500MB

djangosimple-iast

🟩 execution_time seen in benchmark [21.312ms; 21.364ms]; SLO is < 22.250ms
🟩 max_rss_usage seen in benchmark [63.717MB; 63.842MB]; SLO is < 65.500MB

djangosimple-profiler

🟩 execution_time seen in benchmark [15.755ms; 15.836ms]; SLO is < 16.550ms
🟩 max_rss_usage seen in benchmark [50.728MB; 50.840MB]; SLO is < 53.000MB

djangosimple-span-code-origin

🟩 execution_time seen in benchmark [27.070ms; 27.146ms]; SLO is < 28.200ms
🟩 max_rss_usage seen in benchmark [66.060MB; 66.137MB]; SLO is < 68.000MB

djangosimple-tracer

🟩 execution_time seen in benchmark [21.335ms; 21.400ms]; SLO is < 22.700ms
🟩 max_rss_usage seen in benchmark [63.687MB; 63.755MB]; SLO is < 65.500MB

djangosimple-tracer-and-profiler

🟩 execution_time seen in benchmark [23.791ms; 23.856ms]; SLO is < 24.900ms
🟩 max_rss_usage seen in benchmark [65.040MB; 65.193MB]; SLO is < 67.000MB

djangosimple-tracer-no-caches

🟩 execution_time seen in benchmark [18.932ms; 19.003ms]; SLO is < 19.650ms
🟩 max_rss_usage seen in benchmark [63.701MB; 63.701MB]; SLO is < 65.500MB

djangosimple-tracer-no-databases

🟩 execution_time seen in benchmark [19.195ms; 19.262ms]; SLO is < 20.100ms
🟩 max_rss_usage seen in benchmark [63.701MB; 63.701MB]; SLO is < 65.500MB

djangosimple-tracer-no-middleware

🟩 execution_time seen in benchmark [21.092ms; 21.158ms]; SLO is < 22.500ms
🟩 max_rss_usage seen in benchmark [63.687MB; 63.755MB]; SLO is < 65.500MB

djangosimple-tracer-no-templates

🟩 execution_time seen in benchmark [21.174ms; 21.231ms]; SLO is < 22.250ms
🟩 max_rss_usage seen in benchmark [63.687MB; 63.755MB]; SLO is < 65.500MB

errortrackingdjangosimple-errortracking-enabled-all

🟩 execution_time seen in benchmark [18.520ms; 18.572ms]; SLO is < 19.850ms
🟩 max_rss_usage seen in benchmark [63.704MB; 63.816MB]; SLO is < 65.500MB

errortrackingdjangosimple-errortracking-enabled-user

🟩 execution_time seen in benchmark [18.540ms; 18.618ms]; SLO is < 19.400ms
🟩 max_rss_usage seen in benchmark [63.687MB; 63.755MB]; SLO is < 65.500MB

errortrackingdjangosimple-tracer-enabled

🟩 execution_time seen in benchmark [18.536ms; 18.649ms]; SLO is < 19.450ms
🟩 max_rss_usage seen in benchmark [63.696MB; 63.698MB]; SLO is < 65.500MB

errortrackingflasksqli-errortracking-enabled-all

🟩 execution_time seen in benchmark [2.119ms; 2.125ms]; SLO is < 2.300ms
🟩 max_rss_usage seen in benchmark [51.575MB; 51.723MB]; SLO is < 53.000MB

errortrackingflasksqli-errortracking-enabled-user

🟩 execution_time seen in benchmark [2.122ms; 2.131ms]; SLO is < 2.250ms
🟩 max_rss_usage seen in benchmark [51.558MB; 51.701MB]; SLO is < 53.000MB

errortrackingflasksqli-tracer-enabled

🟩 execution_time seen in benchmark [2.118ms; 2.124ms]; SLO is < 2.300ms
🟩 max_rss_usage seen in benchmark [51.134MB; 51.259MB]; SLO is < 53.000MB

flasksimple-appsec-get

🟩 execution_time seen in benchmark [4.611ms; 4.621ms]; SLO is < 4.750ms
🟩 max_rss_usage seen in benchmark [62.932MB; 63.089MB]; SLO is < 64.000MB

flasksimple-appsec-post

🟩 execution_time seen in benchmark [6.623ms; 6.631ms]; SLO is < 6.750ms
🟩 max_rss_usage seen in benchmark [62.969MB; 63.125MB]; SLO is < 64.000MB

flasksimple-appsec-telemetry

🟩 execution_time seen in benchmark [4.614ms; 4.624ms]; SLO is < 4.750ms
🟩 max_rss_usage seen in benchmark [62.915MB; 63.067MB]; SLO is < 64.000MB

flasksimple-debugger

🟩 execution_time seen in benchmark [1.852ms; 1.855ms]; SLO is < 2.000ms
🟩 max_rss_usage seen in benchmark [42.074MB; 42.074MB]; SLO is < 44.000MB

flasksimple-iast-get

🟩 execution_time seen in benchmark [1.858ms; 1.861ms]; SLO is < 2.000ms
🟩 max_rss_usage seen in benchmark [44.468MB; 44.617MB]; SLO is < 45.000MB

flasksimple-profiler

🟩 execution_time seen in benchmark [1.977ms; 1.981ms]; SLO is < 2.100ms
🟩 max_rss_usage seen in benchmark [43.246MB; 43.340MB]; SLO is < 44.000MB

flasksimple-tracer

🟩 execution_time seen in benchmark [3.388ms; 3.394ms]; SLO is < 3.650ms
🟩 max_rss_usage seen in benchmark [51.370MB; 51.495MB]; SLO is < 53.000MB

flasksqli-appsec-enabled

🟩 execution_time seen in benchmark [3.967ms; 3.976ms]; SLO is < 4.200ms
🟩 max_rss_usage seen in benchmark [63.356MB; 63.534MB]; SLO is < 66.000MB

flasksqli-iast-enabled

🟩 execution_time seen in benchmark [2.556ms; 2.581ms]; SLO is < 2.800ms
🟩 max_rss_usage seen in benchmark [56.544MB; 56.663MB]; SLO is < 58.000MB

flasksqli-tracer-enabled

🟩 execution_time seen in benchmark [2.101ms; 2.108ms]; SLO is < 2.250ms
🟩 max_rss_usage seen in benchmark [51.165MB; 51.308MB]; SLO is < 53.000MB

httppropagationextract-all_styles_all_headers

🟩 execution_time seen in benchmark [61.024µs; 61.287µs]; SLO is < 70.000µs
🟩 max_rss_usage seen in benchmark [29.091MB; 29.184MB]; SLO is < 31.000MB

httppropagationextract-b3_headers

🟩 execution_time seen in benchmark [9.889µs; 9.924µs]; SLO is < 20.000µs
🟩 max_rss_usage seen in benchmark [29.101MB; 29.213MB]; SLO is < 31.000MB

httppropagationextract-b3_single_headers

🟩 execution_time seen in benchmark [8.992µs; 9.057µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.084MB; 29.152MB]; SLO is < 31.000MB

httppropagationextract-datadog_tracecontext_tracestate_not_propagated_on_trace_id_no_match

🟩 execution_time seen in benchmark [52.692µs; 53.061µs]; SLO is < 60.000µs
🟩 max_rss_usage seen in benchmark [29.129MB; 29.264MB]; SLO is < 31.000MB

httppropagationextract-datadog_tracecontext_tracestate_propagated_on_trace_id_match

🟩 execution_time seen in benchmark [54.470µs; 54.755µs]; SLO is < 60.000µs
🟩 max_rss_usage seen in benchmark [29.129MB; 29.264MB]; SLO is < 31.000MB

httppropagationextract-empty_headers

🟩 execution_time seen in benchmark [1.588µs; 1.601µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.179MB; 29.332MB]; SLO is < 31.000MB

httppropagationextract-full_t_id_datadog_headers

🟩 execution_time seen in benchmark [17.869µs; 17.991µs]; SLO is < 20.000µs
🟩 max_rss_usage seen in benchmark [29.091MB; 29.184MB]; SLO is < 31.000MB

httppropagationextract-invalid_priority_header

🟩 execution_time seen in benchmark [6.561µs; 6.600µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.101MB; 29.213MB]; SLO is < 31.000MB

httppropagationextract-invalid_span_id_header

🟩 execution_time seen in benchmark [6.492µs; 6.527µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.091MB; 29.184MB]; SLO is < 31.000MB

httppropagationextract-invalid_tags_header

🟩 execution_time seen in benchmark [6.560µs; 6.614µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.129MB; 29.264MB]; SLO is < 31.000MB

httppropagationextract-invalid_trace_id_header

🟩 execution_time seen in benchmark [6.505µs; 6.547µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.097MB; 29.256MB]; SLO is < 31.000MB

httppropagationextract-large_header_no_matches

🟩 execution_time seen in benchmark [27.584µs; 27.679µs]; SLO is < 30.000µs
🟩 max_rss_usage seen in benchmark [29.161MB; 29.310MB]; SLO is < 31.000MB

httppropagationextract-large_valid_headers_all

🟩 execution_time seen in benchmark [28.722µs; 28.909µs]; SLO is < 40.000µs
🟩 max_rss_usage seen in benchmark [29.114MB; 29.239MB]; SLO is < 31.000MB

httppropagationextract-medium_header_no_matches

🟩 execution_time seen in benchmark [9.875µs; 9.931µs]; SLO is < 20.000µs
🟩 max_rss_usage seen in benchmark [29.101MB; 29.213MB]; SLO is < 31.000MB

httppropagationextract-medium_valid_headers_all

🟩 execution_time seen in benchmark [11.273µs; 11.340µs]; SLO is < 20.000µs
🟩 max_rss_usage seen in benchmark [29.129MB; 29.264MB]; SLO is < 31.000MB

httppropagationextract-none_propagation_style

🟩 execution_time seen in benchmark [1.686µs; 1.696µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.091MB; 29.184MB]; SLO is < 31.000MB

httppropagationextract-tracecontext_headers

🟩 execution_time seen in benchmark [27.415µs; 27.637µs]; SLO is < 30.000µs
🟩 max_rss_usage seen in benchmark [29.129MB; 29.264MB]; SLO is < 31.000MB

httppropagationextract-valid_headers_all

🟩 execution_time seen in benchmark [6.504µs; 6.553µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.101MB; 29.213MB]; SLO is < 31.000MB

httppropagationextract-valid_headers_basic

🟩 execution_time seen in benchmark [6.104µs; 6.139µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.114MB; 29.239MB]; SLO is < 31.000MB

httppropagationextract-wsgi_empty_headers

🟩 execution_time seen in benchmark [1.594µs; 1.610µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.091MB; 29.184MB]; SLO is < 31.000MB

httppropagationextract-wsgi_invalid_priority_header

🟩 execution_time seen in benchmark [6.529µs; 6.566µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.101MB; 29.213MB]; SLO is < 31.000MB

httppropagationextract-wsgi_invalid_span_id_header

🟩 execution_time seen in benchmark [1.599µs; 1.611µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.114MB; 29.239MB]; SLO is < 31.000MB

httppropagationextract-wsgi_invalid_tags_header

🟩 execution_time seen in benchmark [6.593µs; 6.654µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.129MB; 29.264MB]; SLO is < 31.000MB

httppropagationextract-wsgi_invalid_trace_id_header

🟩 execution_time seen in benchmark [6.523µs; 6.563µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.129MB; 29.264MB]; SLO is < 31.000MB

httppropagationextract-wsgi_large_header_no_matches

🟩 execution_time seen in benchmark [28.655µs; 28.746µs]; SLO is < 40.000µs
🟩 max_rss_usage seen in benchmark [29.114MB; 29.239MB]; SLO is < 31.000MB

httppropagationextract-wsgi_large_valid_headers_all

🟩 execution_time seen in benchmark [29.770µs; 29.910µs]; SLO is < 40.000µs
🟩 max_rss_usage seen in benchmark [29.114MB; 29.239MB]; SLO is < 31.000MB

httppropagationextract-wsgi_medium_header_no_matches

🟩 execution_time seen in benchmark [10.139µs; 10.211µs]; SLO is < 20.000µs
🟩 max_rss_usage seen in benchmark [29.058MB; 29.178MB]; SLO is < 31.000MB

httppropagationextract-wsgi_medium_valid_headers_all

🟩 execution_time seen in benchmark [11.459µs; 11.543µs]; SLO is < 20.000µs
🟩 max_rss_usage seen in benchmark [29.101MB; 29.213MB]; SLO is < 31.000MB

httppropagationextract-wsgi_valid_headers_all

🟩 execution_time seen in benchmark [6.557µs; 6.596µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.129MB; 29.264MB]; SLO is < 31.000MB

httppropagationextract-wsgi_valid_headers_basic

🟩 execution_time seen in benchmark [6.124µs; 6.165µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.101MB; 29.213MB]; SLO is < 31.000MB

httppropagationinject-ids_only

🟩 execution_time seen in benchmark [11.926µs; 12.046µs]; SLO is < 20.000µs
🟩 max_rss_usage seen in benchmark [29.049MB; 29.147MB]; SLO is < 31.000MB

httppropagationinject-with_all

🟩 execution_time seen in benchmark [24.354µs; 24.612µs]; SLO is < 30.000µs
🟩 max_rss_usage seen in benchmark [29.084MB; 29.152MB]; SLO is < 31.000MB

httppropagationinject-with_dd_origin

🟩 execution_time seen in benchmark [16.929µs; 17.015µs]; SLO is < 20.000µs
🟩 max_rss_usage seen in benchmark [29.084MB; 29.152MB]; SLO is < 31.000MB

httppropagationinject-with_priority_and_origin

🟩 execution_time seen in benchmark [19.037µs; 19.233µs]; SLO is < 20.000µs
🟩 max_rss_usage seen in benchmark [29.091MB; 29.184MB]; SLO is < 31.000MB

httppropagationinject-with_sampling_priority

🟩 execution_time seen in benchmark [14.182µs; 14.288µs]; SLO is < 20.000µs
🟩 max_rss_usage seen in benchmark [29.091MB; 29.184MB]; SLO is < 31.000MB

httppropagationinject-with_tags

🟩 execution_time seen in benchmark [18.539µs; 18.775µs]; SLO is < 20.000µs
🟩 max_rss_usage seen in benchmark [29.161MB; 29.310MB]; SLO is < 31.000MB

httppropagationinject-with_tags_invalid

🟩 execution_time seen in benchmark [20.809µs; 20.933µs]; SLO is < 30.000µs
🟩 max_rss_usage seen in benchmark [29.161MB; 29.310MB]; SLO is < 31.000MB

httppropagationinject-with_tags_max_size

🟩 execution_time seen in benchmark [18.754µs; 18.940µs]; SLO is < 30.000µs
🟩 max_rss_usage seen in benchmark [29.144MB; 29.287MB]; SLO is < 31.000MB

iast_aspects-re_expand_aspect

🟩 execution_time seen in benchmark [33.017µs; 33.263µs]; SLO is < 40.000µs
🟩 max_rss_usage seen in benchmark [34.606MB; 34.718MB]; SLO is < 35.000MB

iast_aspects-re_expand_noaspect

🟩 execution_time seen in benchmark [28.822µs; 29.174µs]; SLO is < 40.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iast_aspects-re_findall_aspect

🟩 execution_time seen in benchmark [3.664µs; 3.690µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iast_aspects-re_findall_noaspect

🟩 execution_time seen in benchmark [1.405µs; 1.420µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iast_aspects-re_finditer_aspect

🟩 execution_time seen in benchmark [5.119µs; 5.141µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iast_aspects-re_finditer_noaspect

🟩 execution_time seen in benchmark [1.405µs; 1.419µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iast_aspects-re_fullmatch_aspect

🟩 execution_time seen in benchmark [3.392µs; 3.418µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iast_aspects-re_fullmatch_noaspect

🟩 execution_time seen in benchmark [1.282µs; 1.295µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iast_aspects-re_group_aspect

🟩 execution_time seen in benchmark [3.443µs; 3.475µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iast_aspects-re_group_noaspect

🟩 execution_time seen in benchmark [1.599µs; 1.621µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iast_aspects-re_groups_aspect

🟩 execution_time seen in benchmark [3.559µs; 3.584µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iast_aspects-re_groups_noaspect

🟩 execution_time seen in benchmark [1.671µs; 1.681µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iast_aspects-re_match_aspect

🟩 execution_time seen in benchmark [3.398µs; 3.425µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iast_aspects-re_match_noaspect

🟩 execution_time seen in benchmark [1.295µs; 1.310µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iast_aspects-re_search_aspect

🟩 execution_time seen in benchmark [3.291µs; 3.311µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iast_aspects-re_search_noaspect

🟩 execution_time seen in benchmark [1.187µs; 1.194µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iast_aspects-re_sub_aspect

🟩 execution_time seen in benchmark [4.671µs; 4.693µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iast_aspects-re_sub_noaspect

🟩 execution_time seen in benchmark [1.521µs; 1.536µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iast_aspects-re_subn_aspect

🟩 execution_time seen in benchmark [4.892µs; 4.922µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.500MB

iast_aspects-re_subn_noaspect

🟩 execution_time seen in benchmark [1.587µs; 1.599µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspects-add_aspect

🟩 execution_time seen in benchmark [327.197ns; 332.596ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iastaspects-add_inplace_aspect

🟩 execution_time seen in benchmark [326.932ns; 329.282ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iastaspects-add_inplace_noaspect

🟩 execution_time seen in benchmark [312.415ns; 315.388ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.606MB; 34.718MB]; SLO is < 35.000MB

iastaspects-add_noaspect

🟩 execution_time seen in benchmark [273.102ns; 278.023ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iastaspects-bytearray_aspect

🟩 execution_time seen in benchmark [1.857µs; 1.873µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspects-bytearray_extend_aspect

🟩 execution_time seen in benchmark [1.374µs; 1.387µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iastaspects-bytearray_extend_noaspect

🟩 execution_time seen in benchmark [608.222ns; 612.696ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iastaspects-bytearray_noaspect

🟩 execution_time seen in benchmark [478.466ns; 482.235ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iastaspects-bytes_aspect

🟩 execution_time seen in benchmark [1.852µs; 1.863µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.606MB; 34.718MB]; SLO is < 35.000MB

iastaspects-bytes_noaspect

🟩 execution_time seen in benchmark [488.399ns; 494.048ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iastaspects-bytesio_aspect

🟩 execution_time seen in benchmark [1.876µs; 1.890µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspects-bytesio_noaspect

🟩 execution_time seen in benchmark [494.152ns; 499.325ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iastaspects-capitalize_aspect

🟩 execution_time seen in benchmark [729.447ns; 735.586ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iastaspects-capitalize_noaspect

🟩 execution_time seen in benchmark [430.218ns; 433.042ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspects-casefold_aspect

🟩 execution_time seen in benchmark [731.770ns; 738.348ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspects-casefold_noaspect

🟩 execution_time seen in benchmark [364.792ns; 369.290ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iastaspects-decode_aspect

🟩 execution_time seen in benchmark [723.516ns; 730.262ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iastaspects-decode_noaspect

🟩 execution_time seen in benchmark [415.799ns; 420.282ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iastaspects-encode_aspect

🟩 execution_time seen in benchmark [704.924ns; 711.903ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspects-encode_noaspect

🟩 execution_time seen in benchmark [400.024ns; 403.499ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iastaspects-format_aspect

🟩 execution_time seen in benchmark [3.412µs; 3.439µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iastaspects-format_map_aspect

🟩 execution_time seen in benchmark [3.195µs; 3.213µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iastaspects-format_map_noaspect

🟩 execution_time seen in benchmark [771.496ns; 778.445ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspects-format_noaspect

🟩 execution_time seen in benchmark [592.865ns; 598.390ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iastaspects-index_aspect

🟩 execution_time seen in benchmark [336.701ns; 340.473ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspects-index_noaspect

🟩 execution_time seen in benchmark [276.030ns; 279.785ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iastaspects-join_aspect

🟩 execution_time seen in benchmark [1.216µs; 1.225µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.606MB; 34.718MB]; SLO is < 35.000MB

iastaspects-join_noaspect

🟩 execution_time seen in benchmark [490.540ns; 495.122ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iastaspects-ljust_aspect

🟩 execution_time seen in benchmark [10.336µs; 10.388µs]; SLO is < 20.000µs
🟩 max_rss_usage seen in benchmark [34.702MB; 34.858MB]; SLO is < 35.500MB

iastaspects-ljust_noaspect

🟩 execution_time seen in benchmark [401.363ns; 407.265ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.619MB; 34.744MB]; SLO is < 35.500MB

iastaspects-lower_aspect

🟩 execution_time seen in benchmark [2.214µs; 2.226µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.500MB

iastaspects-lower_noaspect

🟩 execution_time seen in benchmark [365.340ns; 370.027ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspects-lstrip_aspect

🟩 execution_time seen in benchmark [10.398µs; 10.440µs]; SLO is < 20.000µs
🟩 max_rss_usage seen in benchmark [34.634MB; 34.769MB]; SLO is < 35.500MB

iastaspects-lstrip_noaspect

🟩 execution_time seen in benchmark [381.390ns; 385.375ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspects-modulo_aspect

🟩 execution_time seen in benchmark [578.610ns; 585.819ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspects-modulo_aspect_for_bytearray_bytearray

🟩 execution_time seen in benchmark [1.303µs; 1.324µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iastaspects-modulo_aspect_for_bytes

🟩 execution_time seen in benchmark [747.870ns; 755.360ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspects-modulo_aspect_for_bytes_bytearray

🟩 execution_time seen in benchmark [956.171ns; 966.966ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iastaspects-modulo_noaspect

🟩 execution_time seen in benchmark [624.442ns; 630.511ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iastaspects-replace_aspect

🟩 execution_time seen in benchmark [4.739µs; 4.828µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspects-replace_noaspect

🟩 execution_time seen in benchmark [454.840ns; 459.402ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.500MB

iastaspects-repr_aspect

🟩 execution_time seen in benchmark [904.396ns; 912.237ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iastaspects-repr_noaspect

🟩 execution_time seen in benchmark [411.300ns; 416.435ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspects-rstrip_aspect

🟩 execution_time seen in benchmark [10.302µs; 10.354µs]; SLO is < 20.000µs
🟩 max_rss_usage seen in benchmark [34.619MB; 34.744MB]; SLO is < 35.500MB

iastaspects-rstrip_noaspect

🟩 execution_time seen in benchmark [376.441ns; 380.853ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iastaspects-slice_aspect

🟩 execution_time seen in benchmark [476.381ns; 481.267ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iastaspects-slice_noaspect

🟩 execution_time seen in benchmark [443.044ns; 446.720ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iastaspects-stringio_aspect

🟩 execution_time seen in benchmark [2.188µs; 2.205µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspects-stringio_noaspect

🟩 execution_time seen in benchmark [710.725ns; 720.658ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.619MB; 34.744MB]; SLO is < 35.000MB

iastaspects-strip_aspect

🟩 execution_time seen in benchmark [10.377µs; 10.432µs]; SLO is < 20.000µs
🟩 max_rss_usage seen in benchmark [34.649MB; 34.792MB]; SLO is < 35.500MB

iastaspects-strip_noaspect

🟩 execution_time seen in benchmark [382.682ns; 387.853ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspects-swapcase_aspect

🟩 execution_time seen in benchmark [2.426µs; 2.441µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iastaspects-swapcase_noaspect

🟩 execution_time seen in benchmark [532.945ns; 538.376ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.606MB; 34.718MB]; SLO is < 35.000MB

iastaspects-title_aspect

🟩 execution_time seen in benchmark [2.352µs; 2.372µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iastaspects-title_noaspect

🟩 execution_time seen in benchmark [501.245ns; 506.973ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iastaspects-translate_aspect

🟩 execution_time seen in benchmark [3.299µs; 3.318µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iastaspects-translate_noaspect

🟩 execution_time seen in benchmark [1.042µs; 1.051µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspects-upper_aspect

🟩 execution_time seen in benchmark [2.238µs; 2.257µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iastaspects-upper_noaspect

🟩 execution_time seen in benchmark [369.101ns; 374.755ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iastaspectsospath-ospathbasename_aspect

🟩 execution_time seen in benchmark [4.255µs; 4.346µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iastaspectsospath-ospathbasename_noaspect

🟩 execution_time seen in benchmark [1.078µs; 1.086µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.606MB; 34.718MB]; SLO is < 35.000MB

iastaspectsospath-ospathjoin_aspect

🟩 execution_time seen in benchmark [6.136µs; 6.192µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iastaspectsospath-ospathjoin_noaspect

🟩 execution_time seen in benchmark [2.260µs; 2.275µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.606MB; 34.718MB]; SLO is < 35.000MB

iastaspectsospath-ospathnormcase_aspect

🟩 execution_time seen in benchmark [3.505µs; 3.563µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspectsospath-ospathnormcase_noaspect

🟩 execution_time seen in benchmark [571.368ns; 579.621ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspectsospath-ospathsplit_aspect

🟩 execution_time seen in benchmark [4.813µs; 4.842µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspectsospath-ospathsplit_noaspect

🟩 execution_time seen in benchmark [1.581µs; 1.592µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iastaspectsospath-ospathsplitdrive_aspect

🟩 execution_time seen in benchmark [3.692µs; 3.716µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspectsospath-ospathsplitdrive_noaspect

🟩 execution_time seen in benchmark [693.129ns; 703.524ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iastaspectsospath-ospathsplitext_aspect

🟩 execution_time seen in benchmark [4.540µs; 4.572µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iastaspectsospath-ospathsplitext_noaspect

🟩 execution_time seen in benchmark [1.379µs; 1.392µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspectssplit-rsplit_aspect

🟩 execution_time seen in benchmark [1.473µs; 1.499µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspectssplit-rsplit_noaspect

🟩 execution_time seen in benchmark [578.509ns; 583.554ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastaspectssplit-split_aspect

🟩 execution_time seen in benchmark [1.451µs; 1.481µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iastaspectssplit-split_noaspect

🟩 execution_time seen in benchmark [561.995ns; 565.977ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iastaspectssplit-splitlines_aspect

🟩 execution_time seen in benchmark [1.414µs; 1.428µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.000MB

iastaspectssplit-splitlines_noaspect

🟩 execution_time seen in benchmark [577.799ns; 581.684ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.603MB; 34.603MB]; SLO is < 35.000MB

iastpropagation-no-propagation

🟩 execution_time seen in benchmark [48.934µs; 49.076µs]; SLO is < 60.000µs
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.500MB

iastpropagation-propagation_enabled

🟩 execution_time seen in benchmark [144.262µs; 145.339µs]; SLO is < 160.000µs
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.500MB

iastpropagation-propagation_enabled_100

🟩 execution_time seen in benchmark [1.546ms; 1.558ms]; SLO is < 1.800ms
🟩 max_rss_usage seen in benchmark [34.589MB; 34.657MB]; SLO is < 35.000MB

iastpropagation-propagation_enabled_1000

🟩 execution_time seen in benchmark [29.500ms; 29.683ms]; SLO is < 30.550ms
🟩 max_rss_usage seen in benchmark [34.596MB; 34.689MB]; SLO is < 35.500MB

otelsdkspan-add-event

🟩 execution_time seen in benchmark [40.095ms; 40.311ms]; SLO is < 42.000ms
🟩 max_rss_usage seen in benchmark [31.843MB; 31.937MB]; SLO is < 35.000MB

otelsdkspan-add-link

🟩 execution_time seen in benchmark [36.069ms; 36.244ms]; SLO is < 38.550ms
🟩 max_rss_usage seen in benchmark [31.836MB; 31.904MB]; SLO is < 35.000MB

otelsdkspan-add-metrics

🟩 execution_time seen in benchmark [218.826ms; 220.588ms]; SLO is < 232.000ms
🟩 max_rss_usage seen in benchmark [31.854MB; 31.965MB]; SLO is < 35.000MB

otelsdkspan-add-tags

🟩 execution_time seen in benchmark [212.357ms; 213.622ms]; SLO is < 221.600ms
🟩 max_rss_usage seen in benchmark [31.850MB; 31.850MB]; SLO is < 35.000MB

otelsdkspan-get-context

🟩 execution_time seen in benchmark [28.835ms; 29.032ms]; SLO is < 31.300ms
🟩 max_rss_usage seen in benchmark [31.850MB; 31.850MB]; SLO is < 35.000MB

otelsdkspan-is-recording

🟩 execution_time seen in benchmark [28.973ms; 29.090ms]; SLO is < 31.000ms
🟩 max_rss_usage seen in benchmark [31.850MB; 31.850MB]; SLO is < 35.000MB

otelsdkspan-record-exception

🟩 execution_time seen in benchmark [62.932ms; 63.220ms]; SLO is < 65.850ms
🟩 max_rss_usage seen in benchmark [31.850MB; 31.850MB]; SLO is < 35.000MB

otelsdkspan-set-status

🟩 execution_time seen in benchmark [31.854ms; 32.070ms]; SLO is < 34.150ms
🟩 max_rss_usage seen in benchmark [31.850MB; 31.850MB]; SLO is < 35.000MB

otelsdkspan-start

🟩 execution_time seen in benchmark [28.905ms; 29.137ms]; SLO is < 30.150ms
🟩 max_rss_usage seen in benchmark [31.850MB; 31.850MB]; SLO is < 35.000MB

otelsdkspan-start-finish

🟩 execution_time seen in benchmark [33.770ms; 33.983ms]; SLO is < 35.350ms
🟩 max_rss_usage seen in benchmark [31.843MB; 31.937MB]; SLO is < 35.000MB

otelsdkspan-start-finish-telemetry

🟩 execution_time seen in benchmark [33.544ms; 33.816ms]; SLO is < 35.450ms
🟩 max_rss_usage seen in benchmark [31.850MB; 31.850MB]; SLO is < 35.000MB

otelsdkspan-update-name

🟩 execution_time seen in benchmark [30.973ms; 31.149ms]; SLO is < 33.400ms
🟩 max_rss_usage seen in benchmark [31.850MB; 31.850MB]; SLO is < 35.000MB

otelspan-add-event

🟩 execution_time seen in benchmark [43.568ms; 43.769ms]; SLO is < 47.150ms
🟩 max_rss_usage seen in benchmark [42.173MB; 42.324MB]; SLO is < 42.500MB

otelspan-add-metrics

🟩 execution_time seen in benchmark [317.683ms; 320.326ms]; SLO is < 344.800ms
🟩 max_rss_usage seen in benchmark [558.735MB; 558.735MB]; SLO is < 562.000MB

otelspan-add-tags

🟩 execution_time seen in benchmark [290.708ms; 295.267ms]; SLO is < 314.000ms
🟩 max_rss_usage seen in benchmark [561.328MB; 561.328MB]; SLO is < 563.500MB

otelspan-get-context

🟩 execution_time seen in benchmark [84.033ms; 85.262ms]; SLO is < 92.350ms
🟩 max_rss_usage seen in benchmark [37.162MB; 37.323MB]; SLO is < 38.000MB

otelspan-is-recording

🟩 execution_time seen in benchmark [42.365ms; 42.582ms]; SLO is < 44.500ms
🟩 max_rss_usage seen in benchmark [41.472MB; 41.651MB]; SLO is < 42.000MB

otelspan-record-exception

🟩 execution_time seen in benchmark [59.703ms; 60.733ms]; SLO is < 67.650ms
🟩 max_rss_usage seen in benchmark [37.601MB; 37.753MB]; SLO is < 38.000MB

otelspan-set-status

🟩 execution_time seen in benchmark [47.287ms; 47.483ms]; SLO is < 50.400ms
🟩 max_rss_usage seen in benchmark [41.432MB; 41.578MB]; SLO is < 42.000MB

otelspan-start

🟩 execution_time seen in benchmark [40.671ms; 40.849ms]; SLO is < 43.450ms
🟩 max_rss_usage seen in benchmark [41.534MB; 41.690MB]; SLO is < 42.000MB

otelspan-start-finish

🟩 execution_time seen in benchmark [80.624ms; 80.973ms]; SLO is < 86.000ms
🟩 max_rss_usage seen in benchmark [31.596MB; 31.751MB]; SLO is < 32.000MB

otelspan-start-finish-telemetry

🟩 execution_time seen in benchmark [82.267ms; 82.498ms]; SLO is < 86.000ms
🟩 max_rss_usage seen in benchmark [31.557MB; 31.712MB]; SLO is < 32.000MB

otelspan-update-name

🟩 execution_time seen in benchmark [42.929ms; 43.246ms]; SLO is < 45.150ms
🟩 max_rss_usage seen in benchmark [41.784MB; 41.938MB]; SLO is < 42.500MB

packagespackageforrootmodulemapping-cache_off

🟩 execution_time seen in benchmark [347.096ms; 349.147ms]; SLO is < 354.300ms
🟩 max_rss_usage seen in benchmark [35.423MB; 35.473MB]; SLO is < 38.000MB

packagespackageforrootmodulemapping-cache_on

🟩 execution_time seen in benchmark [381.426ns; 385.217ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [33.889MB; 33.965MB]; SLO is < 38.000MB

packagesupdateimporteddependencies-import_many

🟩 execution_time seen in benchmark [156.042µs; 157.110µs]; SLO is < 170.000µs
🟩 max_rss_usage seen in benchmark [33.902MB; 33.992MB]; SLO is < 35.500MB

packagesupdateimporteddependencies-import_many_cached

🟩 execution_time seen in benchmark [120.460µs; 120.755µs]; SLO is < 130.000µs
🟩 max_rss_usage seen in benchmark [33.994MB; 34.093MB]; SLO is < 35.500MB

packagesupdateimporteddependencies-import_many_stdlib

🟩 execution_time seen in benchmark [1.591ms; 1.614ms]; SLO is < 1.750ms
🟩 max_rss_usage seen in benchmark [34.055MB; 34.182MB]; SLO is < 35.500MB

packagesupdateimporteddependencies-import_many_stdlib_cached

🟩 execution_time seen in benchmark [953.550µs; 975.826µs]; SLO is < 1.100ms
🟩 max_rss_usage seen in benchmark [34.053MB; 34.115MB]; SLO is < 35.500MB

packagesupdateimporteddependencies-import_many_unknown

🟩 execution_time seen in benchmark [825.970µs; 831.638µs]; SLO is < 890.000µs
🟩 max_rss_usage seen in benchmark [33.862MB; 33.984MB]; SLO is < 35.500MB

packagesupdateimporteddependencies-import_many_unknown_cached

🟩 execution_time seen in benchmark [787.509µs; 796.248µs]; SLO is < 870.000µs
🟩 max_rss_usage seen in benchmark [34.001MB; 34.163MB]; SLO is < 35.500MB

packagesupdateimporteddependencies-import_one

🟩 execution_time seen in benchmark [19.931µs; 20.080µs]; SLO is < 30.000µs
🟩 max_rss_usage seen in benchmark [33.932MB; 33.985MB]; SLO is < 35.500MB

packagesupdateimporteddependencies-import_one_cache

🟩 execution_time seen in benchmark [6.285µs; 6.372µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.041MB; 34.098MB]; SLO is < 35.500MB

packagesupdateimporteddependencies-import_one_stdlib

🟩 execution_time seen in benchmark [18.738µs; 18.875µs]; SLO is < 20.000µs
🟩 max_rss_usage seen in benchmark [34.005MB; 34.066MB]; SLO is < 35.500MB

packagesupdateimporteddependencies-import_one_stdlib_cache

🟩 execution_time seen in benchmark [6.203µs; 6.221µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [33.984MB; 34.118MB]; SLO is < 35.500MB

packagesupdateimporteddependencies-import_one_unknown

🟩 execution_time seen in benchmark [45.447µs; 45.769µs]; SLO is < 50.000µs
🟩 max_rss_usage seen in benchmark [33.908MB; 33.962MB]; SLO is < 35.500MB

packagesupdateimporteddependencies-import_one_unknown_cache

🟩 execution_time seen in benchmark [6.250µs; 6.308µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [34.081MB; 34.199MB]; SLO is < 35.500MB

ratelimiter-defaults

🟩 execution_time seen in benchmark [2.353µs; 2.372µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [28.676MB; 28.812MB]; SLO is < 31.000MB

ratelimiter-high_rate_limit

🟩 execution_time seen in benchmark [2.393µs; 2.410µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [28.625MB; 28.745MB]; SLO is < 31.000MB

ratelimiter-long_window

🟩 execution_time seen in benchmark [2.352µs; 2.371µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [28.635MB; 28.775MB]; SLO is < 31.000MB

ratelimiter-low_rate_limit

🟩 execution_time seen in benchmark [2.361µs; 2.377µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [28.676MB; 28.812MB]; SLO is < 31.000MB

ratelimiter-no_rate_limit

🟩 execution_time seen in benchmark [829.881ns; 841.552ns]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [28.690MB; 28.758MB]; SLO is < 31.000MB

ratelimiter-short_window

🟩 execution_time seen in benchmark [2.476µs; 2.493µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [28.608MB; 28.763MB]; SLO is < 31.000MB

recursivecomputation-deep

🟩 execution_time seen in benchmark [308.547ms; 309.323ms]; SLO is < 320.950ms
🟩 max_rss_usage seen in benchmark [30.278MB; 30.278MB]; SLO is < 31.000MB

recursivecomputation-deep-profiled

🟩 execution_time seen in benchmark [344.064ms; 345.791ms]; SLO is < 359.150ms
🟩 max_rss_usage seen in benchmark [34.517MB; 34.610MB]; SLO is < 35.500MB

recursivecomputation-medium

🟩 execution_time seen in benchmark [7.010ms; 7.038ms]; SLO is < 7.400ms
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

recursivecomputation-shallow

🟩 execution_time seen in benchmark [941.701µs; 947.800µs]; SLO is < 1.050ms
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

samplingrules-average_match

🟩 execution_time seen in benchmark [332.037µs; 334.126µs]; SLO is < 350.000µs
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

samplingrules-high_match

🟩 execution_time seen in benchmark [498.720µs; 502.044µs]; SLO is < 550.000µs
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

samplingrules-low_match

🟩 execution_time seen in benchmark [168.400µs; 169.400µs]; SLO is < 190.000µs
🟩 max_rss_usage seen in benchmark [431.751MB; 431.751MB]; SLO is < 432.500MB

samplingrules-very_low_match

🟩 execution_time seen in benchmark [8.586ms; 8.634ms]; SLO is < 9.150ms
🟩 max_rss_usage seen in benchmark [54.872MB; 54.920MB]; SLO is < 55.000MB

sethttpmeta-all-disabled

🟩 execution_time seen in benchmark [12.135µs; 12.193µs]; SLO is < 20.000µs
🟩 max_rss_usage seen in benchmark [29.491MB; 29.491MB]; SLO is < 31.000MB

sethttpmeta-all-enabled

🟩 execution_time seen in benchmark [41.791µs; 42.020µs]; SLO is < 50.000µs
🟩 max_rss_usage seen in benchmark [29.491MB; 29.491MB]; SLO is < 31.000MB

sethttpmeta-collectipvariant_exists

🟩 execution_time seen in benchmark [42.489µs; 42.685µs]; SLO is < 50.000µs
🟩 max_rss_usage seen in benchmark [29.491MB; 29.491MB]; SLO is < 31.000MB

sethttpmeta-no-collectipvariant

🟩 execution_time seen in benchmark [41.663µs; 41.872µs]; SLO is < 50.000µs
🟩 max_rss_usage seen in benchmark [29.522MB; 29.657MB]; SLO is < 31.000MB

sethttpmeta-no-useragentvariant

🟩 execution_time seen in benchmark [40.649µs; 40.868µs]; SLO is < 50.000µs
🟩 max_rss_usage seen in benchmark [29.477MB; 29.545MB]; SLO is < 31.000MB

sethttpmeta-obfuscation-no-query

🟩 execution_time seen in benchmark [42.421µs; 42.710µs]; SLO is < 50.000µs
🟩 max_rss_usage seen in benchmark [29.491MB; 29.491MB]; SLO is < 31.000MB

sethttpmeta-obfuscation-regular-case-explicit-query

🟩 execution_time seen in benchmark [78.863µs; 79.125µs]; SLO is < 90.000µs
🟩 max_rss_usage seen in benchmark [29.870MB; 29.938MB]; SLO is < 31.000MB

sethttpmeta-obfuscation-regular-case-implicit-query

🟩 execution_time seen in benchmark [79.334µs; 79.541µs]; SLO is < 90.000µs
🟩 max_rss_usage seen in benchmark [29.870MB; 29.938MB]; SLO is < 31.000MB

sethttpmeta-obfuscation-send-querystring-disabled

🟩 execution_time seen in benchmark [156.850µs; 157.090µs]; SLO is < 170.000µs
🟩 max_rss_usage seen in benchmark [29.870MB; 29.938MB]; SLO is < 31.000MB

sethttpmeta-obfuscation-worst-case-explicit-query

🟩 execution_time seen in benchmark [150.731µs; 151.002µs]; SLO is < 160.000µs
🟩 max_rss_usage seen in benchmark [29.884MB; 29.884MB]; SLO is < 31.000MB

sethttpmeta-obfuscation-worst-case-implicit-query

🟩 execution_time seen in benchmark [157.505µs; 157.828µs]; SLO is < 170.000µs
🟩 max_rss_usage seen in benchmark [29.915MB; 30.050MB]; SLO is < 31.000MB

sethttpmeta-useragentvariant_exists_1

🟩 execution_time seen in benchmark [41.300µs; 41.615µs]; SLO is < 50.000µs
🟩 max_rss_usage seen in benchmark [29.494MB; 29.606MB]; SLO is < 31.000MB

sethttpmeta-useragentvariant_exists_2

🟩 execution_time seen in benchmark [42.371µs; 42.573µs]; SLO is < 50.000µs
🟩 max_rss_usage seen in benchmark [29.484MB; 29.577MB]; SLO is < 31.000MB

sethttpmeta-useragentvariant_exists_3

🟩 execution_time seen in benchmark [41.923µs; 42.184µs]; SLO is < 50.000µs
🟩 max_rss_usage seen in benchmark [29.491MB; 29.491MB]; SLO is < 31.000MB

sethttpmeta-useragentvariant_not_exists_1

🟩 execution_time seen in benchmark [41.250µs; 41.481µs]; SLO is < 50.000µs
🟩 max_rss_usage seen in benchmark [29.484MB; 29.577MB]; SLO is < 31.000MB

sethttpmeta-useragentvariant_not_exists_2

🟩 execution_time seen in benchmark [41.159µs; 41.360µs]; SLO is < 50.000µs
🟩 max_rss_usage seen in benchmark [29.477MB; 29.545MB]; SLO is < 31.000MB

span-add-event

🟩 execution_time seen in benchmark [22.794ms; 23.171ms]; SLO is < 26.200ms
🟩 max_rss_usage seen in benchmark [48.466MB; 48.490MB]; SLO is < 49.000MB

span-add-metrics

🟩 execution_time seen in benchmark [89.710ms; 90.845ms]; SLO is < 98.350ms
🟩 max_rss_usage seen in benchmark [614.896MB; 614.896MB]; SLO is < 961.000MB

span-add-tags

🟩 execution_time seen in benchmark [147.512ms; 149.469ms]; SLO is < 168.550ms
🟩 max_rss_usage seen in benchmark [613.732MB; 613.876MB]; SLO is < 962.500MB

span-get-context

🟩 execution_time seen in benchmark [21.230ms; 21.599ms]; SLO is < 23.700ms
🟩 max_rss_usage seen in benchmark [47.270MB; 47.301MB]; SLO is < 47.500MB

span-is-recording

🟩 execution_time seen in benchmark [21.212ms; 21.610ms]; SLO is < 23.900ms
🟩 max_rss_usage seen in benchmark [47.284MB; 47.308MB]; SLO is < 47.500MB

span-record-exception

🟩 execution_time seen in benchmark [40.602ms; 40.789ms]; SLO is < 44.500ms
🟩 max_rss_usage seen in benchmark [40.305MB; 40.343MB]; SLO is < 40.500MB

span-set-status

🟩 execution_time seen in benchmark [22.916ms; 23.308ms]; SLO is < 26.000ms
🟩 max_rss_usage seen in benchmark [47.264MB; 47.294MB]; SLO is < 47.500MB

span-start

🟩 execution_time seen in benchmark [20.861ms; 21.192ms]; SLO is < 23.500ms
🟩 max_rss_usage seen in benchmark [47.275MB; 47.305MB]; SLO is < 47.500MB

span-start-finish

🟩 execution_time seen in benchmark [49.768ms; 49.974ms]; SLO is < 52.500ms
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

span-start-finish-telemetry

🟩 execution_time seen in benchmark [51.344ms; 51.522ms]; SLO is < 55.300ms
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

span-start-finish-traceid128

🟩 execution_time seen in benchmark [53.276ms; 53.474ms]; SLO is < 56.050ms
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

span-start-traceid128

🟩 execution_time seen in benchmark [21.623ms; 22.010ms]; SLO is < 24.600ms
🟩 max_rss_usage seen in benchmark [47.294MB; 47.319MB]; SLO is < 47.500MB

span-update-name

🟩 execution_time seen in benchmark [21.682ms; 22.071ms]; SLO is < 24.100ms
🟩 max_rss_usage seen in benchmark [47.800MB; 47.814MB]; SLO is < 48.000MB

telemetryaddmetric-1-count-metric-1-times

🟩 execution_time seen in benchmark [3.137µs; 3.152µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

telemetryaddmetric-1-count-metrics-100-times

🟩 execution_time seen in benchmark [216.275µs; 217.680µs]; SLO is < 240.000µs
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

telemetryaddmetric-1-distribution-metric-1-times

🟩 execution_time seen in benchmark [2.927µs; 2.949µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

telemetryaddmetric-1-distribution-metrics-100-times

🟩 execution_time seen in benchmark [190.916µs; 191.802µs]; SLO is < 210.000µs
🟩 max_rss_usage seen in benchmark [29.055MB; 29.101MB]; SLO is < 31.000MB

telemetryaddmetric-1-gauge-metric-1-times

🟩 execution_time seen in benchmark [2.263µs; 2.299µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

telemetryaddmetric-1-gauge-metrics-100-times

🟩 execution_time seen in benchmark [123.865µs; 124.273µs]; SLO is < 140.000µs
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

telemetryaddmetric-1-rate-metric-1-times

🟩 execution_time seen in benchmark [3.209µs; 3.259µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

telemetryaddmetric-1-rate-metrics-100-times

🟩 execution_time seen in benchmark [216.317µs; 217.124µs]; SLO is < 230.000µs
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

telemetryaddmetric-100-count-metrics-100-times

🟩 execution_time seen in benchmark [21.526ms; 21.674ms]; SLO is < 22.500ms
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

telemetryaddmetric-100-distribution-metrics-100-times

🟩 execution_time seen in benchmark [1.999ms; 2.011ms]; SLO is < 2.100ms
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

telemetryaddmetric-100-gauge-metrics-100-times

🟩 execution_time seen in benchmark [1.287ms; 1.294ms]; SLO is < 1.400ms
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

telemetryaddmetric-100-rate-metrics-100-times

🟩 execution_time seen in benchmark [2.226ms; 2.246ms]; SLO is < 2.400ms
🟩 max_rss_usage seen in benchmark [29.055MB; 29.101MB]; SLO is < 31.000MB

telemetryaddmetric-flush-1-metric

🟩 execution_time seen in benchmark [4.339µs; 4.408µs]; SLO is < 10.000µs
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

telemetryaddmetric-flush-100-metrics

🟩 execution_time seen in benchmark [180.579µs; 181.721µs]; SLO is < 200.000µs
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

telemetryaddmetric-flush-1000-metrics

🟩 execution_time seen in benchmark [2.177ms; 2.185ms]; SLO is < 2.350ms
🟩 max_rss_usage seen in benchmark [30.235MB; 30.281MB]; SLO is < 31.000MB

tracer-large

🟩 execution_time seen in benchmark [29.514ms; 29.882ms]; SLO is < 32.950ms
🟩 max_rss_usage seen in benchmark [30.278MB; 30.278MB]; SLO is < 31.000MB

tracer-medium

🟩 execution_time seen in benchmark [2.914ms; 2.942ms]; SLO is < 3.200ms
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

tracer-small

🟩 execution_time seen in benchmark [329.130µs; 330.387µs]; SLO is < 370.000µs
🟩 max_rss_usage seen in benchmark [29.098MB; 29.098MB]; SLO is < 31.000MB

Legend:

🟩 pass
🟥 breach
🟨 warning
(unstable) unstable

Note: All comparisons are against the mean unless a different statistic (e.g., p95) is explicitly shown.

Yun-Kim

Looks good! Two small suggestions

ddtrace/llmobs/_integrations/utils.py

releasenotes/notes/litellm-scrub-metadata-135109a6a5324111.yaml

Co-authored-by: Yun Kim <[email protected]>

github-actions · 2025-07-21T19:52:00Z

The backport to 3.10 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-3.10 3.10
# Navigate to the new working tree
cd .worktrees/backport-3.10
# Create a new branch
git switch --create backport-14067-to-3.10
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 e31f11f61e04d44f0046bf7f10c8aeeb9322c358
# Push it to GitHub
git push --set-upstream origin backport-14067-to-3.10
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-3.10

Then, create a pull request where the base branch is 3.10 and the compare/head branch is backport-14067-to-3.10.

…rgs (#14067) We received a customer ticket explaining that `vertex_credentials` was being collected as a metadata field on LLM spans emitted by the LiteLLM integration. This PR addresses that issue and safeguards against potentially other sensitive information being exposed by explicitly selecting a subset of kwargs to attach to the LLM span's metadata. Keys were chosen based on the arguments passed into the LiteLLM SDK [completion](https://github.com/BerriAI/litellm/blob/main/litellm/main.py#L874-L917) method and [text_completion](https://github.com/BerriAI/litellm/blob/main/litellm/main.py#L4446-L4497) method. I verified that this PR does resolve the issue. The following script was run to produce this [trace](https://app.datadoghq.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZgeNigRWrccCwAAABhBWmdlTmlnUkFBQmM2eEc4M1pUNUFBQUEAAAAkMDE5ODFlMzYtNDNiMi00NGQ1LWJlMTUtNzk3MDUxZTNmNTBhAAAALQ%22%7D%5D&spanId=1740319997238873855&start=1752852637158&end=1752853537158&paused=false) where the metadata field does not contain `vertex_credentials`. ``` from litellm import completion import json file_path = '/path/to/credentials' with open(file_path, 'r') as file: vertex_credentials = json.load(file) vertex_credentials_json = json.dumps(vertex_credentials) response = completion( model="vertex_ai/gemini-1.5-flash", messages=[{ "content": "Hello, how are you?","role": "user"}], vertex_credentials=vertex_credentials_json, stream=True ) for chunk in response: print(chunk) ``` - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) --------- Co-authored-by: Yun Kim <[email protected]> (cherry picked from commit e31f11f)

…rgs (#14067) We received a customer ticket explaining that `vertex_credentials` was being collected as a metadata field on LLM spans emitted by the LiteLLM integration. This PR addresses that issue and safeguards against potentially other sensitive information being exposed by explicitly selecting a subset of kwargs to attach to the LLM span's metadata. Keys were chosen based on the arguments passed into the LiteLLM SDK [completion](https://github.com/BerriAI/litellm/blob/main/litellm/main.py#L874-L917) method and [text_completion](https://github.com/BerriAI/litellm/blob/main/litellm/main.py#L4446-L4497) method. I verified that this PR does resolve the issue. The following script was run to produce this [trace](https://app.datadoghq.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZgeNigRWrccCwAAABhBWmdlTmlnUkFBQmM2eEc4M1pUNUFBQUEAAAAkMDE5ODFlMzYtNDNiMi00NGQ1LWJlMTUtNzk3MDUxZTNmNTBhAAAALQ%22%7D%5D&spanId=1740319997238873855&start=1752852637158&end=1752853537158&paused=false) where the metadata field does not contain `vertex_credentials`. ``` from litellm import completion import json file_path = '/path/to/credentials' with open(file_path, 'r') as file: vertex_credentials = json.load(file) vertex_credentials_json = json.dumps(vertex_credentials) response = completion( model="vertex_ai/gemini-1.5-flash", messages=[{ "content": "Hello, how are you?","role": "user"}], vertex_credentials=vertex_credentials_json, stream=True ) for chunk in response: print(chunk) ``` ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) --------- Co-authored-by: Yun Kim <[email protected]>

select metadata keys to tag from kwargs

42e5c5d

Merge branch 'main' into nicole-cybul/redact-litellm-params

dac423d

add release note

83fe788

ncybul marked this pull request as ready for review July 21, 2025 13:47

ncybul requested review from a team as code owners July 21, 2025 13:47

ncybul requested review from duncanista and sabrenner July 21, 2025 13:47

ncybul changed the title ~~fix(litellm): select metadata keys to tag from litellm kwargs~~ fix(litellm): [MLOS-182] select metadata keys to tag from litellm kwargs Jul 21, 2025

Yun-Kim approved these changes Jul 21, 2025

View reviewed changes

ddtrace/llmobs/_integrations/utils.py Show resolved Hide resolved

releasenotes/notes/litellm-scrub-metadata-135109a6a5324111.yaml Outdated Show resolved Hide resolved

Update releasenotes/notes/litellm-scrub-metadata-135109a6a5324111.yaml

4986965

Co-authored-by: Yun Kim <[email protected]>

ncybul merged commit e31f11f into main Jul 21, 2025
468 checks passed

ncybul deleted the nicole-cybul/redact-litellm-params branch July 21, 2025 19:00

ncybul added the backport 3.10 label Jul 21, 2025

fix(litellm): [MLOS-182] select metadata keys to tag from litellm kwargs #14067

fix(litellm): [MLOS-182] select metadata keys to tag from litellm kwargs #14067

Uh oh!

Conversation

ncybul commented Jul 18, 2025 • edited by Yun-Kim Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Reviewer Checklist

Uh oh!

github-actions bot commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bootstrap import analysis

Summary

Import time breakdown

Uh oh!

pr-commenter bot commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Performance SLOs

coreapiscenario-context_with_data_listeners

coreapiscenario-context_with_data_no_listeners

coreapiscenario-context_with_data_only_all_listeners

coreapiscenario-get_item_exists

coreapiscenario-get_item_missing

coreapiscenario-set_item

djangosimple-appsec

djangosimple-exception-replay-enabled

djangosimple-iast

djangosimple-profiler

djangosimple-span-code-origin

djangosimple-tracer

djangosimple-tracer-and-profiler

djangosimple-tracer-no-caches

djangosimple-tracer-no-databases

djangosimple-tracer-no-middleware

djangosimple-tracer-no-templates

errortrackingdjangosimple-errortracking-enabled-all

errortrackingdjangosimple-errortracking-enabled-user

errortrackingdjangosimple-tracer-enabled

errortrackingflasksqli-errortracking-enabled-all

errortrackingflasksqli-errortracking-enabled-user

errortrackingflasksqli-tracer-enabled

flasksimple-appsec-get

flasksimple-appsec-post

flasksimple-appsec-telemetry

flasksimple-debugger

flasksimple-iast-get

flasksimple-profiler

flasksimple-tracer

flasksqli-appsec-enabled

flasksqli-iast-enabled

flasksqli-tracer-enabled

httppropagationextract-all_styles_all_headers

httppropagationextract-b3_headers

httppropagationextract-b3_single_headers

httppropagationextract-datadog_tracecontext_tracestate_not_propagated_on_trace_id_no_match

httppropagationextract-datadog_tracecontext_tracestate_propagated_on_trace_id_match

httppropagationextract-empty_headers

httppropagationextract-full_t_id_datadog_headers

httppropagationextract-invalid_priority_header

httppropagationextract-invalid_span_id_header

httppropagationextract-invalid_tags_header

httppropagationextract-invalid_trace_id_header

httppropagationextract-large_header_no_matches

httppropagationextract-large_valid_headers_all

httppropagationextract-medium_header_no_matches

httppropagationextract-medium_valid_headers_all

httppropagationextract-none_propagation_style

httppropagationextract-tracecontext_headers

httppropagationextract-valid_headers_all

httppropagationextract-valid_headers_basic

httppropagationextract-wsgi_empty_headers

httppropagationextract-wsgi_invalid_priority_header

httppropagationextract-wsgi_invalid_span_id_header

httppropagationextract-wsgi_invalid_tags_header

httppropagationextract-wsgi_invalid_trace_id_header

httppropagationextract-wsgi_large_header_no_matches

httppropagationextract-wsgi_large_valid_headers_all

ncybul commented Jul 18, 2025 •

edited by Yun-Kim

Loading

github-actions bot commented Jul 18, 2025 •

edited

Loading

github-actions bot commented Jul 18, 2025 •

edited

Loading

pr-commenter bot commented Jul 18, 2025 •

edited

Loading