Skip to content

Add tests for llmobs #9395

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Aug 28, 2025
Merged

Add tests for llmobs #9395

merged 1 commit into from
Aug 28, 2025
+296 −2

Conversation

nayeem-kamal
Copy link
Contributor

@nayeem-kamal nayeem-kamal commented Aug 20, 2025 •
edited
Loading

What Does This Do

Resolves the issue of non-static methods in LLMObs and adds tests to prevent future issues of this nature.

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@datadog-datadog-prod-us1
Copy link
Contributor

datadog-datadog-prod-us1 bot commented Aug 20, 2025 •
edited
Loading

🎯 Code Coverage
• Patch Coverage: 100.00%
• Total Coverage: 57.44% (-0.10%)

View detailed report

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 9456358 | Docs | Was this helpful? Give us feedback!

@pr-commenter
Copy link

pr-commenter bot commented Aug 20, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nayeem-kamal/llmobs-tests1
git_commit_date 1755718248 1755718258
git_commit_sha c2e8852 9456358
release_version 1.53.0-SNAPSHOT~c2e8852e57 1.53.0-SNAPSHOT~9456358eee
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1755720112 1755720112
ci_job_id 1090678324 1090678324
ci_pipeline_id 74299119 74299119
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-2-n2mf7mj0 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-2-n2mf7mj0 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 47 metrics, 12 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.53.0-SNAPSHOT~9456358eee, baseline=1.53.0-SNAPSHOT~c2e8852e57

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.048 s) : 0, 1048398
Total [baseline] (10.677 s) : 0, 10677191
Agent [candidate] (1.046 s) : 0, 1045526
Total [candidate] (10.774 s) : 0, 10774151
section appsec
Agent [baseline] (1.224 s) : 0, 1223643
Total [baseline] (10.801 s) : 0, 10801086
Agent [candidate] (1.24 s) : 0, 1239779
Total [candidate] (10.841 s) : 0, 10840595
section iast
Agent [baseline] (1.183 s) : 0, 1182894
Total [baseline] (10.947 s) : 0, 10947115
Agent [candidate] (1.179 s) : 0, 1178666
Total [candidate] (10.879 s) : 0, 10878932
section profiling
Agent [baseline] (1.206 s) : 0, 1205751
Total [baseline] (10.992 s) : 0, 10991908
Agent [candidate] (1.196 s) : 0, 1196371
Total [candidate] (10.841 s) : 0, 10841403
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.048 s -
Agent appsec 1.224 s 175.246 ms (16.7%)
Agent iast 1.183 s 134.497 ms (12.8%)
Agent profiling 1.206 s 157.353 ms (15.0%)
Total tracing 10.677 s -
Total appsec 10.801 s 123.895 ms (1.2%)
Total iast 10.947 s 269.924 ms (2.5%)
Total profiling 10.992 s 314.717 ms (2.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.046 s -
Agent appsec 1.24 s 194.254 ms (18.6%)
Agent iast 1.179 s 133.141 ms (12.7%)
Agent profiling 1.196 s 150.845 ms (14.4%)
Total tracing 10.774 s -
Total appsec 10.841 s 66.445 ms (0.6%)
Total iast 10.879 s 104.781 ms (1.0%)
Total profiling 10.841 s 67.252 ms (0.6%) 
gantt
    title petclinic - break down per module: candidate=1.53.0-SNAPSHOT~9456358eee, baseline=1.53.0-SNAPSHOT~c2e8852e57

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.456 ms) : 0, 1456
crashtracking [candidate] (1.445 ms) : 0, 1445
BytebuddyAgent [baseline] (733.369 ms) : 0, 733369
BytebuddyAgent [candidate] (732.302 ms) : 0, 732302
GlobalTracer [baseline] (242.777 ms) : 0, 242777
GlobalTracer [candidate] (242.718 ms) : 0, 242718
AppSec [baseline] (30.086 ms) : 0, 30086
AppSec [candidate] (29.982 ms) : 0, 29982
Debugger [baseline] (6.094 ms) : 0, 6094
Debugger [candidate] (6.029 ms) : 0, 6029
Remote Config [baseline] (669.526 µs) : 0, 670
Remote Config [candidate] (662.136 µs) : 0, 662
Telemetry [baseline] (12.888 ms) : 0, 12888
Telemetry [candidate] (11.401 ms) : 0, 11401
section appsec
crashtracking [baseline] (1.456 ms) : 0, 1456
crashtracking [candidate] (1.471 ms) : 0, 1471
BytebuddyAgent [baseline] (756.078 ms) : 0, 756078
BytebuddyAgent [candidate] (766.956 ms) : 0, 766956
GlobalTracer [baseline] (235.708 ms) : 0, 235708
GlobalTracer [candidate] (238.935 ms) : 0, 238935
IAST [baseline] (23.534 ms) : 0, 23534
IAST [candidate] (24.042 ms) : 0, 24042
AppSec [baseline] (169.425 ms) : 0, 169425
AppSec [candidate] (169.872 ms) : 0, 169872
Debugger [baseline] (6.434 ms) : 0, 6434
Debugger [candidate] (7.27 ms) : 0, 7270
Remote Config [baseline] (626.399 µs) : 0, 626
Remote Config [candidate] (635.325 µs) : 0, 635
Telemetry [baseline] (9.222 ms) : 0, 9222
Telemetry [candidate] (9.248 ms) : 0, 9248
section iast
crashtracking [baseline] (1.46 ms) : 0, 1460
crashtracking [candidate] (1.462 ms) : 0, 1462
BytebuddyAgent [baseline] (853.452 ms) : 0, 853452
BytebuddyAgent [candidate] (850.241 ms) : 0, 850241
GlobalTracer [baseline] (234.111 ms) : 0, 234111
GlobalTracer [candidate] (234.087 ms) : 0, 234087
IAST [baseline] (31.102 ms) : 0, 31102
IAST [candidate] (30.335 ms) : 0, 30335
AppSec [baseline] (26.111 ms) : 0, 26111
AppSec [candidate] (26.051 ms) : 0, 26051
Debugger [baseline] (6.682 ms) : 0, 6682
Debugger [candidate] (6.612 ms) : 0, 6612
Remote Config [baseline] (593.835 µs) : 0, 594
Remote Config [candidate] (610.083 µs) : 0, 610
Telemetry [baseline] (8.294 ms) : 0, 8294
Telemetry [candidate] (8.273 ms) : 0, 8273
section profiling
crashtracking [baseline] (1.426 ms) : 0, 1426
crashtracking [candidate] (1.429 ms) : 0, 1429
BytebuddyAgent [baseline] (768.303 ms) : 0, 768303
BytebuddyAgent [candidate] (762.561 ms) : 0, 762561
GlobalTracer [baseline] (223.722 ms) : 0, 223722
GlobalTracer [candidate] (222.055 ms) : 0, 222055
AppSec [baseline] (30.349 ms) : 0, 30349
AppSec [candidate] (30.073 ms) : 0, 30073
Debugger [baseline] (6.319 ms) : 0, 6319
Debugger [candidate] (6.261 ms) : 0, 6261
Remote Config [baseline] (711.101 µs) : 0, 711
Remote Config [candidate] (697.761 µs) : 0, 698
Telemetry [baseline] (15.587 ms) : 0, 15587
Telemetry [candidate] (16.353 ms) : 0, 16353
ProfilingAgent [baseline] (109.446 ms) : 0, 109446
ProfilingAgent [candidate] (107.463 ms) : 0, 107463
Profiling [baseline] (110.1 ms) : 0, 110100
Profiling [candidate] (108.123 ms) : 0, 108123
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.53.0-SNAPSHOT~9456358eee, baseline=1.53.0-SNAPSHOT~c2e8852e57

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.049 s) : 0, 1049414
Total [baseline] (8.617 s) : 0, 8617279
Agent [candidate] (1.049 s) : 0, 1048951
Total [candidate] (8.588 s) : 0, 8588209
section iast
Agent [baseline] (1.178 s) : 0, 1178100
Total [baseline] (9.316 s) : 0, 9315860
Agent [candidate] (1.183 s) : 0, 1183209
Total [candidate] (9.323 s) : 0, 9322886
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.049 s -
Agent iast 1.178 s 128.686 ms (12.3%)
Total tracing 8.617 s -
Total iast 9.316 s 698.581 ms (8.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.049 s -
Agent iast 1.183 s 134.258 ms (12.8%)
Total tracing 8.588 s -
Total iast 9.323 s 734.677 ms (8.6%) 
gantt
    title insecure-bank - break down per module: candidate=1.53.0-SNAPSHOT~9456358eee, baseline=1.53.0-SNAPSHOT~c2e8852e57

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.466 ms) : 0, 1466
crashtracking [candidate] (1.451 ms) : 0, 1451
BytebuddyAgent [baseline] (734.506 ms) : 0, 734506
BytebuddyAgent [candidate] (732.956 ms) : 0, 732956
GlobalTracer [baseline] (242.898 ms) : 0, 242898
GlobalTracer [candidate] (242.968 ms) : 0, 242968
AppSec [baseline] (30.209 ms) : 0, 30209
AppSec [candidate] (30.137 ms) : 0, 30137
Debugger [baseline] (6.103 ms) : 0, 6103
Debugger [candidate] (6.079 ms) : 0, 6079
Remote Config [baseline] (663.856 µs) : 0, 664
Remote Config [candidate] (659.88 µs) : 0, 660
Telemetry [baseline] (12.443 ms) : 0, 12443
Telemetry [candidate] (13.726 ms) : 0, 13726
section iast
crashtracking [baseline] (1.441 ms) : 0, 1441
crashtracking [candidate] (1.443 ms) : 0, 1443
BytebuddyAgent [baseline] (850.173 ms) : 0, 850173
BytebuddyAgent [candidate] (853.864 ms) : 0, 853864
GlobalTracer [baseline] (233.562 ms) : 0, 233562
GlobalTracer [candidate] (234.807 ms) : 0, 234807
IAST [baseline] (25.27 ms) : 0, 25270
IAST [candidate] (31.22 ms) : 0, 31220
AppSec [baseline] (28.633 ms) : 0, 28633
AppSec [candidate] (26.149 ms) : 0, 26149
Debugger [baseline] (9.047 ms) : 0, 9047
Debugger [candidate] (5.795 ms) : 0, 5795
Remote Config [baseline] (600.132 µs) : 0, 600
Remote Config [candidate] (625.545 µs) : 0, 626
Telemetry [baseline] (8.321 ms) : 0, 8321
Telemetry [candidate] (8.313 ms) : 0, 8313
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nayeem-kamal/llmobs-tests1
git_commit_date 1755718248 1755718258
git_commit_sha c2e8852 9456358
release_version 1.53.0-SNAPSHOT~c2e8852e57 1.53.0-SNAPSHOT~9456358eee
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1755719788 1755719788
ci_job_id 1090678325 1090678325
ci_pipeline_id 74299119 74299119
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-3-67jwe54g 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-3-67jwe54g 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 1 performance regressions! Performance is the same for 9 metrics, 12 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:insecure-bank:iast:high_load better
[-739.379µs; -394.398µs] or [-7.590%; -4.049%]		 unstable
[-30.861op/s; +89.361op/s] or [-6.474%; +18.745%]		 9.174ms 505.969op/s 9.741ms 476.719op/s
scenario:load:petclinic:no_agent:high_load worse
[+1.553ms; +2.201ms] or [+4.271%; +6.055%]		 unstable
[-15.366op/s; +2.691op/s] or [-11.947%; +2.092%]		 38.228ms 122.275op/s 36.351ms 128.613op/s
scenario:load:petclinic:tracing:high_load better
[-2.116ms; -1.300ms] or [-4.684%; -2.878%]		 unstable
[-1.824op/s; +12.674op/s] or [-1.761%; +12.235%]		 43.460ms 109.013op/s 45.168ms 103.588op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~9456358eee, baseline=1.53.0-SNAPSHOT~c2e8852e57
    dateFormat X
    axisFormat %s
section baseline
no_agent (36.351 ms) : 36067, 36636
.   : milestone, 36351,
appsec (46.364 ms) : 45951, 46776
.   : milestone, 46364,
code_origins (44.74 ms) : 44342, 45139
.   : milestone, 44740,
iast (45.22 ms) : 44819, 45621
.   : milestone, 45220,
profiling (46.631 ms) : 46211, 47052
.   : milestone, 46631,
tracing (45.168 ms) : 44784, 45552
.   : milestone, 45168,
section candidate
no_agent (38.228 ms) : 37911, 38545
.   : milestone, 38228,
appsec (47.509 ms) : 47086, 47933
.   : milestone, 47509,
code_origins (45.67 ms) : 45259, 46080
.   : milestone, 45670,
iast (46.449 ms) : 46050, 46849
.   : milestone, 46449,
profiling (45.383 ms) : 44960, 45806
.   : milestone, 45383,
tracing (43.46 ms) : 43086, 43835
.   : milestone, 43460,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 36.351 ms [36.067 ms, 36.636 ms] -
appsec 46.364 ms [45.951 ms, 46.776 ms] 10.012 ms (27.5%)
code_origins 44.74 ms [44.342 ms, 45.139 ms] 8.389 ms (23.1%)
iast 45.22 ms [44.819 ms, 45.621 ms] 8.869 ms (24.4%)
profiling 46.631 ms [46.211 ms, 47.052 ms] 10.28 ms (28.3%)
tracing 45.168 ms [44.784 ms, 45.552 ms] 8.817 ms (24.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 38.228 ms [37.911 ms, 38.545 ms] -
appsec 47.509 ms [47.086 ms, 47.933 ms] 9.281 ms (24.3%)
code_origins 45.67 ms [45.259 ms, 46.08 ms] 7.441 ms (19.5%)
iast 46.449 ms [46.05 ms, 46.849 ms] 8.221 ms (21.5%)
profiling 45.383 ms [44.96 ms, 45.806 ms] 7.155 ms (18.7%)
tracing 43.46 ms [43.086 ms, 43.835 ms] 5.232 ms (13.7%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~9456358eee, baseline=1.53.0-SNAPSHOT~c2e8852e57
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.246 ms) : 4193, 4299
.   : milestone, 4246,
iast (9.741 ms) : 9580, 9903
.   : milestone, 9741,
iast_FULL (14.306 ms) : 14021, 14590
.   : milestone, 14306,
iast_GLOBAL (10.464 ms) : 10278, 10650
.   : milestone, 10464,
profiling (8.617 ms) : 8487, 8747
.   : milestone, 8617,
tracing (7.755 ms) : 7645, 7866
.   : milestone, 7755,
section candidate
no_agent (4.273 ms) : 4225, 4321
.   : milestone, 4273,
iast (9.174 ms) : 9015, 9333
.   : milestone, 9174,
iast_FULL (13.747 ms) : 13474, 14020
.   : milestone, 13747,
iast_GLOBAL (10.466 ms) : 10281, 10650
.   : milestone, 10466,
profiling (8.589 ms) : 8457, 8721
.   : milestone, 8589,
tracing (7.554 ms) : 7440, 7667
.   : milestone, 7554,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.246 ms [4.193 ms, 4.299 ms] -
iast 9.741 ms [9.58 ms, 9.903 ms] 5.495 ms (129.4%)
iast_FULL 14.306 ms [14.021 ms, 14.59 ms] 10.06 ms (236.9%)
iast_GLOBAL 10.464 ms [10.278 ms, 10.65 ms] 6.218 ms (146.4%)
profiling 8.617 ms [8.487 ms, 8.747 ms] 4.371 ms (103.0%)
tracing 7.755 ms [7.645 ms, 7.866 ms] 3.51 ms (82.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.273 ms [4.225 ms, 4.321 ms] -
iast 9.174 ms [9.015 ms, 9.333 ms] 4.901 ms (114.7%)
iast_FULL 13.747 ms [13.474 ms, 14.02 ms] 9.474 ms (221.7%)
iast_GLOBAL 10.466 ms [10.281 ms, 10.65 ms] 6.193 ms (144.9%)
profiling 8.589 ms [8.457 ms, 8.721 ms] 4.316 ms (101.0%)
tracing 7.554 ms [7.44 ms, 7.667 ms] 3.281 ms (76.8%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nayeem-kamal/llmobs-tests1
git_commit_date 1755718248 1755718258
git_commit_sha c2e8852 9456358
release_version 1.53.0-SNAPSHOT~c2e8852e57 1.53.0-SNAPSHOT~9456358eee
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1755720274 1755720274
ci_job_id 1090678326 1090678326
ci_pipeline_id 74299119 74299119
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-c2zxj77k 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-c2zxj77k 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~9456358eee, baseline=1.53.0-SNAPSHOT~c2e8852e57
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.476 ms) : 1464, 1487
.   : milestone, 1476,
appsec (3.674 ms) : 3456, 3892
.   : milestone, 3674,
iast (2.202 ms) : 2139, 2264
.   : milestone, 2202,
iast_GLOBAL (2.246 ms) : 2183, 2309
.   : milestone, 2246,
profiling (2.051 ms) : 2000, 2101
.   : milestone, 2051,
tracing (2.017 ms) : 1968, 2065
.   : milestone, 2017,
section candidate
no_agent (1.478 ms) : 1466, 1489
.   : milestone, 1478,
appsec (3.655 ms) : 3435, 3875
.   : milestone, 3655,
iast (2.204 ms) : 2141, 2266
.   : milestone, 2204,
iast_GLOBAL (2.243 ms) : 2180, 2306
.   : milestone, 2243,
profiling (2.054 ms) : 2003, 2104
.   : milestone, 2054,
tracing (2.03 ms) : 1981, 2079
.   : milestone, 2030,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.476 ms [1.464 ms, 1.487 ms] -
appsec 3.674 ms [3.456 ms, 3.892 ms] 2.198 ms (149.0%)
iast 2.202 ms [2.139 ms, 2.264 ms] 726.012 µs (49.2%)
iast_GLOBAL 2.246 ms [2.183 ms, 2.309 ms] 770.353 µs (52.2%)
profiling 2.051 ms [2.0 ms, 2.101 ms] 575.304 µs (39.0%)
tracing 2.017 ms [1.968 ms, 2.065 ms] 541.027 µs (36.7%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.478 ms [1.466 ms, 1.489 ms] -
appsec 3.655 ms [3.435 ms, 3.875 ms] 2.177 ms (147.4%)
iast 2.204 ms [2.141 ms, 2.266 ms] 725.971 µs (49.1%)
iast_GLOBAL 2.243 ms [2.18 ms, 2.306 ms] 765.221 µs (51.8%)
profiling 2.054 ms [2.003 ms, 2.104 ms] 576.254 µs (39.0%)
tracing 2.03 ms [1.981 ms, 2.079 ms] 552.044 µs (37.4%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~9456358eee, baseline=1.53.0-SNAPSHOT~c2e8852e57
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.023 s) : 15023000, 15023000
.   : milestone, 15023000,
appsec (14.893 s) : 14893000, 14893000
.   : milestone, 14893000,
iast (18.632 s) : 18632000, 18632000
.   : milestone, 18632000,
iast_GLOBAL (18.325 s) : 18325000, 18325000
.   : milestone, 18325000,
profiling (15.352 s) : 15352000, 15352000
.   : milestone, 15352000,
tracing (14.731 s) : 14731000, 14731000
.   : milestone, 14731000,
section candidate
no_agent (15.53 s) : 15530000, 15530000
.   : milestone, 15530000,
appsec (14.676 s) : 14676000, 14676000
.   : milestone, 14676000,
iast (17.989 s) : 17989000, 17989000
.   : milestone, 17989000,
iast_GLOBAL (18.095 s) : 18095000, 18095000
.   : milestone, 18095000,
profiling (15.006 s) : 15006000, 15006000
.   : milestone, 15006000,
tracing (14.849 s) : 14849000, 14849000
.   : milestone, 14849000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.023 s [15.023 s, 15.023 s] -
appsec 14.893 s [14.893 s, 14.893 s] -130.0 ms (-0.9%)
iast 18.632 s [18.632 s, 18.632 s] 3.609 s (24.0%)
iast_GLOBAL 18.325 s [18.325 s, 18.325 s] 3.302 s (22.0%)
profiling 15.352 s [15.352 s, 15.352 s] 329.0 ms (2.2%)
tracing 14.731 s [14.731 s, 14.731 s] -292.0 ms (-1.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.53 s [15.53 s, 15.53 s] -
appsec 14.676 s [14.676 s, 14.676 s] -854.0 ms (-5.5%)
iast 17.989 s [17.989 s, 17.989 s] 2.459 s (15.8%)
iast_GLOBAL 18.095 s [18.095 s, 18.095 s] 2.565 s (16.5%)
profiling 15.006 s [15.006 s, 15.006 s] -524.0 ms (-3.4%)
tracing 14.849 s [14.849 s, 14.849 s] -681.0 ms (-4.4%)

@nayeem-kamal nayeem-kamal marked this pull request as ready for review August 26, 2025 14:20
@nayeem-kamal nayeem-kamal requested review from a team as code owners August 26, 2025 14:20
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 26, 2025 •
edited
Loading

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@nayeem-kamal nayeem-kamal requested review from gary-huang and PerfectSlayer and removed request for PerfectSlayer August 26, 2025 14:21
@nayeem-kamal nayeem-kamal mentioned this pull request Aug 26, 2025
Open
@nayeem-kamal nayeem-kamal changed the title Added tests for llmobs Add tests for llmobs Aug 26, 2025
@nayeem-kamal nayeem-kamal added tag: no release notes Changes to exclude from release notes comp: mlobs ML Observability (LLMObs) labels Aug 26, 2025
@nayeem-kamal nayeem-kamal merged commit d07fee9 into master Aug 28, 2025
505 of 509 checks passed
@nayeem-kamal nayeem-kamal deleted the nayeem-kamal/llmobs-tests1 branch August 28, 2025 18:15
@github-actions github-actions bot added this to the 1.53.0 milestone Aug 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gary-huang gary-huang gary-huang approved these changes

@PerfectSlayer PerfectSlayer Awaiting requested review from PerfectSlayer

Assignees
No one assigned
Labels
comp: mlobs ML Observability (LLMObs) tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.53.0
Development

Successfully merging this pull request may close these issues.
2 participants
@nayeem-kamal @gary-huang