-
Notifications
You must be signed in to change notification settings - Fork 312
Upload code coverage reports to Datadog #9425
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Upload code coverage reports to Datadog #9425
Conversation
🎯 Code Coverage 🔗 Commit SHA: c0d713c | Docs | Was this helpful? Give us feedback! |
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 13 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.53.0-SNAPSHOT~c0d713c226, baseline=1.53.0-SNAPSHOT~8799a82b0a
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.047 s) : 0, 1046768
Total [baseline] (10.723 s) : 0, 10722565
Agent [candidate] (1.043 s) : 0, 1043361
Total [candidate] (10.734 s) : 0, 10734430
section appsec
Agent [baseline] (1.221 s) : 0, 1221397
Total [baseline] (10.862 s) : 0, 10862195
Agent [candidate] (1.224 s) : 0, 1224239
Total [candidate] (10.823 s) : 0, 10823245
section iast
Agent [baseline] (1.179 s) : 0, 1178936
Total [baseline] (10.922 s) : 0, 10921724
Agent [candidate] (1.187 s) : 0, 1186925
Total [candidate] (10.925 s) : 0, 10925448
section profiling
Agent [baseline] (1.197 s) : 0, 1196840
Total [baseline] (10.923 s) : 0, 10923049
Agent [candidate] (1.209 s) : 0, 1208977
Total [candidate] (10.992 s) : 0, 10992256
gantt
title petclinic - break down per module: candidate=1.53.0-SNAPSHOT~c0d713c226, baseline=1.53.0-SNAPSHOT~8799a82b0a
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.453 ms) : 0, 1453
crashtracking [candidate] (1.455 ms) : 0, 1455
BytebuddyAgent [baseline] (731.343 ms) : 0, 731343
BytebuddyAgent [candidate] (731.091 ms) : 0, 731091
GlobalTracer [baseline] (241.93 ms) : 0, 241930
GlobalTracer [candidate] (241.737 ms) : 0, 241737
AppSec [baseline] (30.098 ms) : 0, 30098
AppSec [candidate] (30.029 ms) : 0, 30029
Debugger [baseline] (6.035 ms) : 0, 6035
Debugger [candidate] (6.042 ms) : 0, 6042
Remote Config [baseline] (667.13 µs) : 0, 667
Remote Config [candidate] (654.56 µs) : 0, 655
Telemetry [baseline] (14.279 ms) : 0, 14279
Telemetry [candidate] (11.366 ms) : 0, 11366
section appsec
crashtracking [baseline] (1.445 ms) : 0, 1445
crashtracking [candidate] (1.44 ms) : 0, 1440
BytebuddyAgent [baseline] (754.505 ms) : 0, 754505
BytebuddyAgent [candidate] (756.219 ms) : 0, 756219
GlobalTracer [baseline] (235.043 ms) : 0, 235043
GlobalTracer [candidate] (235.314 ms) : 0, 235314
AppSec [baseline] (169.428 ms) : 0, 169428
AppSec [candidate] (169.393 ms) : 0, 169393
Debugger [baseline] (6.499 ms) : 0, 6499
Debugger [candidate] (6.541 ms) : 0, 6541
Remote Config [baseline] (667.361 µs) : 0, 667
Remote Config [candidate] (640.53 µs) : 0, 641
Telemetry [baseline] (9.235 ms) : 0, 9235
Telemetry [candidate] (9.927 ms) : 0, 9927
IAST [baseline] (23.556 ms) : 0, 23556
IAST [candidate] (23.644 ms) : 0, 23644
section iast
crashtracking [baseline] (1.448 ms) : 0, 1448
crashtracking [candidate] (1.456 ms) : 0, 1456
BytebuddyAgent [baseline] (850.829 ms) : 0, 850829
BytebuddyAgent [candidate] (856.998 ms) : 0, 856998
GlobalTracer [baseline] (233.217 ms) : 0, 233217
GlobalTracer [candidate] (235.072 ms) : 0, 235072
AppSec [baseline] (26.696 ms) : 0, 26696
AppSec [candidate] (27.897 ms) : 0, 27897
Debugger [baseline] (5.765 ms) : 0, 5765
Debugger [candidate] (6.655 ms) : 0, 6655
Remote Config [baseline] (595.431 µs) : 0, 595
Remote Config [candidate] (606.27 µs) : 0, 606
Telemetry [baseline] (8.326 ms) : 0, 8326
Telemetry [candidate] (8.358 ms) : 0, 8358
IAST [baseline] (30.988 ms) : 0, 30988
IAST [candidate] (28.667 ms) : 0, 28667
section profiling
crashtracking [baseline] (1.429 ms) : 0, 1429
crashtracking [candidate] (1.463 ms) : 0, 1463
BytebuddyAgent [baseline] (761.859 ms) : 0, 761859
BytebuddyAgent [candidate] (771.738 ms) : 0, 771738
GlobalTracer [baseline] (222.279 ms) : 0, 222279
GlobalTracer [candidate] (223.648 ms) : 0, 223648
AppSec [baseline] (30.957 ms) : 0, 30957
AppSec [candidate] (31.161 ms) : 0, 31161
Debugger [baseline] (6.255 ms) : 0, 6255
Debugger [candidate] (6.337 ms) : 0, 6337
Remote Config [baseline] (691.259 µs) : 0, 691
Remote Config [candidate] (707.222 µs) : 0, 707
Telemetry [baseline] (15.451 ms) : 0, 15451
Telemetry [candidate] (15.488 ms) : 0, 15488
ProfilingAgent [baseline] (108.289 ms) : 0, 108289
ProfilingAgent [candidate] (107.998 ms) : 0, 107998
Profiling [baseline] (108.944 ms) : 0, 108944
Profiling [candidate] (108.636 ms) : 0, 108636
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.53.0-SNAPSHOT~c0d713c226, baseline=1.53.0-SNAPSHOT~8799a82b0a
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.045 s) : 0, 1044681
Total [baseline] (8.622 s) : 0, 8622303
Agent [candidate] (1.049 s) : 0, 1048794
Total [candidate] (8.647 s) : 0, 8647487
section iast
Agent [baseline] (1.187 s) : 0, 1186978
Total [baseline] (9.335 s) : 0, 9334835
Agent [candidate] (1.197 s) : 0, 1197160
Total [candidate] (9.334 s) : 0, 9334103
gantt
title insecure-bank - break down per module: candidate=1.53.0-SNAPSHOT~c0d713c226, baseline=1.53.0-SNAPSHOT~8799a82b0a
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.457 ms) : 0, 1457
crashtracking [candidate] (1.449 ms) : 0, 1449
BytebuddyAgent [baseline] (732.648 ms) : 0, 732648
BytebuddyAgent [candidate] (735.247 ms) : 0, 735247
GlobalTracer [baseline] (242.658 ms) : 0, 242658
GlobalTracer [candidate] (242.873 ms) : 0, 242873
AppSec [baseline] (30.143 ms) : 0, 30143
AppSec [candidate] (30.129 ms) : 0, 30129
Debugger [baseline] (6.059 ms) : 0, 6059
Debugger [candidate] (6.053 ms) : 0, 6053
Remote Config [baseline] (668.869 µs) : 0, 669
Remote Config [candidate] (664.892 µs) : 0, 665
Telemetry [baseline] (10.071 ms) : 0, 10071
Telemetry [candidate] (11.405 ms) : 0, 11405
section iast
crashtracking [baseline] (1.464 ms) : 0, 1464
crashtracking [candidate] (1.482 ms) : 0, 1482
BytebuddyAgent [baseline] (857.403 ms) : 0, 857403
BytebuddyAgent [candidate] (867.381 ms) : 0, 867381
GlobalTracer [baseline] (233.975 ms) : 0, 233975
GlobalTracer [candidate] (234.114 ms) : 0, 234114
AppSec [baseline] (30.355 ms) : 0, 30355
AppSec [candidate] (28.381 ms) : 0, 28381
Debugger [baseline] (5.759 ms) : 0, 5759
Debugger [candidate] (6.67 ms) : 0, 6670
Remote Config [baseline] (599.192 µs) : 0, 599
Remote Config [candidate] (614.097 µs) : 0, 614
Telemetry [baseline] (8.431 ms) : 0, 8431
Telemetry [candidate] (8.447 ms) : 0, 8447
IAST [baseline] (27.867 ms) : 0, 27867
IAST [candidate] (28.803 ms) : 0, 28803
LoadParameters
See matching parameters
SummaryFound 1 performance improvements and 2 performance regressions! Performance is the same for 9 metrics, 12 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~c0d713c226, baseline=1.53.0-SNAPSHOT~8799a82b0a
dateFormat X
axisFormat %s
section baseline
no_agent (4.503 ms) : 4453, 4554
. : milestone, 4503,
iast (9.249 ms) : 9099, 9399
. : milestone, 9249,
iast_FULL (14.178 ms) : 13892, 14463
. : milestone, 14178,
iast_GLOBAL (10.34 ms) : 10159, 10521
. : milestone, 10340,
profiling (9.161 ms) : 9015, 9307
. : milestone, 9161,
tracing (7.649 ms) : 7542, 7756
. : milestone, 7649,
section candidate
no_agent (4.418 ms) : 4362, 4473
. : milestone, 4418,
iast (9.486 ms) : 9333, 9639
. : milestone, 9486,
iast_FULL (13.838 ms) : 13558, 14118
. : milestone, 13838,
iast_GLOBAL (10.573 ms) : 10388, 10758
. : milestone, 10573,
profiling (8.7 ms) : 8561, 8840
. : milestone, 8700,
tracing (7.609 ms) : 7493, 7725
. : milestone, 7609,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~c0d713c226, baseline=1.53.0-SNAPSHOT~8799a82b0a
dateFormat X
axisFormat %s
section baseline
no_agent (37.477 ms) : 37179, 37775
. : milestone, 37477,
appsec (47.035 ms) : 46611, 47459
. : milestone, 47035,
code_origins (44.021 ms) : 43642, 44400
. : milestone, 44021,
iast (45.879 ms) : 45482, 46275
. : milestone, 45879,
profiling (48.329 ms) : 47867, 48792
. : milestone, 48329,
tracing (43.579 ms) : 43212, 43946
. : milestone, 43579,
section candidate
no_agent (38.295 ms) : 37989, 38602
. : milestone, 38295,
appsec (48.156 ms) : 47713, 48598
. : milestone, 48156,
code_origins (44.892 ms) : 44506, 45278
. : milestone, 44892,
iast (45.723 ms) : 45320, 46126
. : milestone, 45723,
profiling (50.21 ms) : 49682, 50738
. : milestone, 50210,
tracing (45.931 ms) : 45552, 46310
. : milestone, 45931,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~c0d713c226, baseline=1.53.0-SNAPSHOT~8799a82b0a
dateFormat X
axisFormat %s
section baseline
no_agent (1.471 ms) : 1460, 1483
. : milestone, 1471,
appsec (3.669 ms) : 3452, 3886
. : milestone, 3669,
iast (2.201 ms) : 2138, 2265
. : milestone, 2201,
iast_GLOBAL (2.23 ms) : 2168, 2293
. : milestone, 2230,
profiling (2.47 ms) : 2307, 2632
. : milestone, 2470,
tracing (2.017 ms) : 1968, 2065
. : milestone, 2017,
section candidate
no_agent (1.481 ms) : 1469, 1492
. : milestone, 1481,
appsec (3.668 ms) : 3450, 3887
. : milestone, 3668,
iast (2.196 ms) : 2133, 2259
. : milestone, 2196,
iast_GLOBAL (2.234 ms) : 2172, 2297
. : milestone, 2234,
profiling (2.025 ms) : 1975, 2075
. : milestone, 2025,
tracing (2.004 ms) : 1956, 2053
. : milestone, 2004,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~c0d713c226, baseline=1.53.0-SNAPSHOT~8799a82b0a
dateFormat X
axisFormat %s
section baseline
no_agent (15.027 s) : 15027000, 15027000
. : milestone, 15027000,
appsec (14.726 s) : 14726000, 14726000
. : milestone, 14726000,
iast (18.687 s) : 18687000, 18687000
. : milestone, 18687000,
iast_GLOBAL (18.034 s) : 18034000, 18034000
. : milestone, 18034000,
profiling (15.405 s) : 15405000, 15405000
. : milestone, 15405000,
tracing (14.882 s) : 14882000, 14882000
. : milestone, 14882000,
section candidate
no_agent (15.693 s) : 15693000, 15693000
. : milestone, 15693000,
appsec (14.893 s) : 14893000, 14893000
. : milestone, 14893000,
iast (18.85 s) : 18850000, 18850000
. : milestone, 18850000,
iast_GLOBAL (18.053 s) : 18053000, 18053000
. : milestone, 18053000,
profiling (15.301 s) : 15301000, 15301000
. : milestone, 15301000,
tracing (14.969 s) : 14969000, 14969000
. : milestone, 14969000,
|
2e80687
to
8093ae4
Compare
Hi! 👋 Thanks for your pull request! 🎉 To help us review it, please make sure to:
If you need help, please check our contributing guidelines. |
What Does This Do
Implements uploading code coverage reports to Datadog's Code Coverage directly from the tracer.
If Jacoco coverage data is detected, it is either uploaded as is or, if there were tests skipped by ITR, is augmented with the coverage of the skipped tests (received from the backend) and is uploaded as an LCOV report (as it is simpler that reconstruct a Jacoco report).
Motivation
Simplify onboarding to Code Coverage for existing Test Optimization customers.
Contributor Checklist
type:
and (comp:
orinst:
) labels in addition to any usefull labelsclose
,fix
or any linking keywords when referencing an issue.Use
solves
instead, and assign the PR milestone to the issueJira ticket: SDTEST-2562