Skip to content

Conversation

@jbachorik
Copy link
Collaborator

@jbachorik jbachorik commented Nov 27, 2025

A trivial fix to always prepare reports.

The test reports are uploaded only on failure but they were generated only on success (default condition).

Note: The perf regression was introduced in TLS priming and #303 is working on dealing with it


Unsure? Have a question? Request a review!

@pr-commenter
Copy link

pr-commenter bot commented Nov 27, 2025

Benchmarks [x86_64 wall]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.3 1.35.0-jb_gh_fix_prepare_reports-SNAPSHOT
See matching parameters
Baseline Candidate
alloc off off
cpu off off
iterations 5 5
java "11.0.28" "11.0.28"
memleak off off
modes wall wall
wall on on

Summary

Found 0 performance improvements and 2 performance regressions! Performance is the same for 14 metrics, 22 unstable metrics.

scenario Δ mean execution_time Δ mean rss
scenario:renaissance:finagle-http worse
[+694.705ms; +1077.295ms] or [+2.641%; +4.096%]
unstable
[-251.724MB; +381.300MB] or [-18.599%; +28.173%]
scenario:renaissance:gauss-mix worse
[+752.473ms; +847.527ms] or [+4.155%; +4.680%]
unstable
[-408.439MB; +493.532MB] or [-34.193%; +41.317%]

@pr-commenter
Copy link

pr-commenter bot commented Nov 27, 2025

Benchmarks [x86_64 memleak]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.3 1.35.0-jb_gh_fix_prepare_reports-SNAPSHOT
See matching parameters
Baseline Candidate
alloc off off
cpu off off
iterations 5 5
java "11.0.28" "11.0.28"
memleak on on
modes memleak memleak
wall off off

Summary

Found 0 performance improvements and 3 performance regressions! Performance is the same for 12 metrics, 23 unstable metrics.

scenario Δ mean execution_time Δ mean rss
scenario:renaissance:finagle-http worse
[+432.251ms; +1319.749ms] or [+1.645%; +5.023%]
unstable
[-272.760MB; +370.201MB] or [-19.732%; +26.781%]
scenario:renaissance:fj-kmeans worse
[+568.118ms; +659.882ms] or [+2.441%; +2.835%]
unstable
[-242.741MB; +361.333MB] or [-23.100%; +34.385%]
scenario:renaissance:gauss-mix worse
[+653.506ms; +898.494ms] or [+3.596%; +4.944%]
unstable
[-392.710MB; +510.059MB] or [-33.049%; +42.925%]

@pr-commenter
Copy link

pr-commenter bot commented Nov 27, 2025

Benchmarks [x86_64 cpu]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.3 1.35.0-jb_gh_fix_prepare_reports-SNAPSHOT
See matching parameters
Baseline Candidate
alloc off off
cpu on on
iterations 5 5
java "11.0.28" "11.0.28"
memleak off off
modes cpu cpu
wall off off

Summary

Found 0 performance improvements and 3 performance regressions! Performance is the same for 12 metrics, 23 unstable metrics.

scenario Δ mean execution_time Δ mean rss
scenario:renaissance:fj-kmeans worse
[+455.544ms; +676.456ms] or [+1.953%; +2.900%]
unstable
[-250.306MB; +355.032MB] or [-23.688%; +33.599%]
scenario:renaissance:scala-kmeans worse
[+616.709ms; +975.291ms] or [+2.674%; +4.229%]
unstable
[-228.075MB; +341.992MB] or [-22.987%; +34.468%]
scenario:renaissance:gauss-mix worse
[+735.962ms; +900.038ms] or [+4.058%; +4.963%]
unstable
[-395.503MB; +507.412MB] or [-33.241%; +42.647%]

@pr-commenter
Copy link

pr-commenter bot commented Nov 27, 2025

Benchmarks [x86_64 alloc]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.3 1.35.0-jb_gh_fix_prepare_reports-SNAPSHOT
See matching parameters
Baseline Candidate
alloc on on
cpu off off
iterations 5 5
java "11.0.28" "11.0.28"
memleak off off
modes alloc alloc
wall off off

Summary

Found 0 performance improvements and 3 performance regressions! Performance is the same for 12 metrics, 23 unstable metrics.

scenario Δ mean execution_time Δ mean rss
scenario:renaissance:finagle-http worse
[+550.806ms; +873.194ms] or [+2.068%; +3.278%]
unstable
[-255.883MB; +386.464MB] or [-18.645%; +28.160%]
scenario:renaissance:fj-kmeans worse
[+512.701ms; +643.299ms] or [+2.199%; +2.759%]
unstable
[-249.124MB; +357.214MB] or [-23.559%; +33.781%]
scenario:renaissance:gauss-mix worse
[+666.254ms; +809.746ms] or [+3.658%; +4.446%]
unstable
[-394.423MB; +511.377MB] or [-33.089%; +42.901%]

@pr-commenter
Copy link

pr-commenter bot commented Nov 27, 2025

Benchmarks [aarch64 wall]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.3 1.35.0-jb_gh_fix_prepare_reports-SNAPSHOT
See matching parameters
Baseline Candidate
alloc off off
cpu off off
iterations 5 5
java "11.0.28" "11.0.28"
memleak off off
modes wall wall
wall on on

Summary

Found 0 performance improvements and 2 performance regressions! Performance is the same for 14 metrics, 22 unstable metrics.

scenario Δ mean execution_time Δ mean rss
scenario:renaissance:future-genetic worse
[+638.487ms; +857.513ms] or [+4.295%; +5.768%]
unstable
[-248.797MB; +559.682MB] or [-29.314%; +65.944%]
scenario:renaissance:dec-tree worse
[+456.026ms; +719.974ms] or [+1.515%; +2.392%]
unstable
[-185.337MB; +426.694MB] or [-13.756%; +31.670%]

@pr-commenter
Copy link

pr-commenter bot commented Nov 27, 2025

Benchmarks [x86_64 cpu,wall,alloc,memleak]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.3 1.35.0-jb_gh_fix_prepare_reports-SNAPSHOT
See matching parameters
Baseline Candidate
alloc on on
cpu on on
iterations 5 5
java "11.0.28" "11.0.28"
memleak on on
modes cpu,wall,alloc,memleak cpu,wall,alloc,memleak
wall on on

Summary

Found 0 performance improvements and 3 performance regressions! Performance is the same for 13 metrics, 22 unstable metrics.

scenario Δ mean execution_time Δ mean rss
scenario:renaissance:fj-kmeans worse
[+371.562ms; +540.438ms] or [+1.585%; +2.305%]
unstable
[-243.328MB; +361.213MB] or [-23.134%; +34.341%]
scenario:renaissance:scala-kmeans worse
[+742.729ms; +977.271ms] or [+3.229%; +4.249%]
unstable
[-227.451MB; +343.006MB] or [-22.922%; +34.568%]
scenario:renaissance:gauss-mix worse
[+677.982ms; +870.018ms] or [+3.729%; +4.785%]
unstable
[-398.895MB; +507.438MB] or [-33.358%; +42.435%]

@pr-commenter
Copy link

pr-commenter bot commented Nov 27, 2025

Benchmarks [x86_64 cpu,wall]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.3 1.35.0-jb_gh_fix_prepare_reports-SNAPSHOT
See matching parameters
Baseline Candidate
alloc off off
cpu on on
iterations 5 5
java "11.0.28" "11.0.28"
memleak off off
modes cpu,wall cpu,wall
wall on on

Summary

Found 0 performance improvements and 3 performance regressions! Performance is the same for 12 metrics, 23 unstable metrics.

scenario Δ mean execution_time Δ mean rss
scenario:renaissance:dotty worse
[+815.054ms; +944.946ms] or [+2.197%; +2.548%]
unstable
[-180.922MB; +311.317MB] or [-12.635%; +21.742%]
scenario:renaissance:scala-kmeans worse
[+535.231ms; +1032.769ms] or [+2.320%; +4.476%]
unstable
[-229.548MB; +340.038MB] or [-23.136%; +34.272%]
scenario:renaissance:gauss-mix worse
[+793.207ms; +886.793ms] or [+4.380%; +4.897%]
unstable
[-398.629MB; +505.504MB] or [-33.415%; +42.374%]

@pr-commenter
Copy link

pr-commenter bot commented Nov 27, 2025

Benchmarks [aarch64 cpu,wall]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.3 1.35.0-jb_gh_fix_prepare_reports-SNAPSHOT
See matching parameters
Baseline Candidate
alloc off off
cpu on on
iterations 5 5
java "11.0.28" "11.0.28"
memleak off off
modes cpu,wall cpu,wall
wall on on

Summary

Found 0 performance improvements and 3 performance regressions! Performance is the same for 13 metrics, 22 unstable metrics.

scenario Δ mean execution_time Δ mean rss
scenario:renaissance:future-genetic worse
[+329.726ms; +754.274ms] or [+2.187%; +5.003%]
unstable
[-273.520MB; +494.396MB] or [-30.337%; +54.835%]
scenario:renaissance:chi-square worse
[+754.041ms; +929.959ms] or [+4.757%; +5.867%]
unstable
[-361.146MB; +466.884MB] or [-32.836%; +42.450%]
scenario:renaissance:mnemonics worse
[+0.335s; +1.905s] or [+1.570%; +8.927%]
unstable
[-235.312MB; +355.791MB] or [-22.999%; +34.775%]

@pr-commenter
Copy link

pr-commenter bot commented Nov 27, 2025

Benchmarks [aarch64 memleak,alloc]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.3 1.35.0-jb_gh_fix_prepare_reports-SNAPSHOT
See matching parameters
Baseline Candidate
alloc on on
cpu off off
iterations 5 5
java "11.0.28" "11.0.28"
memleak on on
modes memleak,alloc memleak,alloc
wall off off

Summary

Found 0 performance improvements and 5 performance regressions! Performance is the same for 12 metrics, 21 unstable metrics.

scenario Δ mean execution_time Δ mean rss
scenario:renaissance:finagle-chirper worse
[+0.664s; +1.904s] or [+2.016%; +5.784%]
unstable
[-220.446MB; +336.713MB] or [-15.667%; +23.930%]
scenario:renaissance:future-genetic worse
[+358.662ms; +725.338ms] or [+2.379%; +4.811%]
unstable
[-272.858MB; +494.715MB] or [-30.288%; +54.915%]
scenario:renaissance:akka-uct worse
[+0.491s; +1.589s] or [+1.656%; +5.361%]
unstable
[-179.786MB; +348.870MB] or [-14.810%; +28.738%]
scenario:renaissance:scala-kmeans worse
[+435.775ms; +704.225ms] or [+1.811%; +2.927%]
unstable
[-226.260MB; +339.319MB] or [-23.084%; +34.618%]
scenario:renaissance:mnemonics worse
[+0.584s; +1.644s] or [+2.734%; +7.705%]
unstable
[-231.620MB; +360.130MB] or [-22.659%; +35.231%]

@pr-commenter
Copy link

pr-commenter bot commented Nov 27, 2025

Benchmarks [aarch64 cpu,wall,alloc,memleak]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.3 1.35.0-jb_gh_fix_prepare_reports-SNAPSHOT
See matching parameters
Baseline Candidate
alloc on on
cpu on on
iterations 5 5
java "11.0.28" "11.0.28"
memleak on on
modes cpu,wall,alloc,memleak cpu,wall,alloc,memleak
wall on on

Summary

Found 0 performance improvements and 3 performance regressions! Performance is the same for 13 metrics, 22 unstable metrics.

scenario Δ mean execution_time Δ mean rss
scenario:renaissance:future-genetic worse
[+438.038ms; +769.962ms] or [+2.920%; +5.132%]
unstable
[-261.672MB; +526.871MB] or [-29.805%; +60.011%]
scenario:renaissance:chi-square worse
[+509.512ms; +1466.488ms] or [+3.291%; +9.471%]
unstable
[-350.009MB; +499.127MB] or [-32.326%; +46.098%]
scenario:renaissance:fj-kmeans worse
[+493.592ms; +786.408ms] or [+2.347%; +3.739%]
unstable
[-244.530MB; +351.879MB] or [-23.607%; +33.971%]

@pr-commenter
Copy link

pr-commenter bot commented Nov 27, 2025

Benchmarks [aarch64 alloc]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.3 1.35.0-jb_gh_fix_prepare_reports-SNAPSHOT
See matching parameters
Baseline Candidate
alloc on on
cpu off off
iterations 5 5
java "11.0.28" "11.0.28"
memleak off off
modes alloc alloc
wall off off

Summary

Found 0 performance improvements and 2 performance regressions! Performance is the same for 15 metrics, 21 unstable metrics.

scenario Δ mean execution_time Δ mean rss
scenario:renaissance:future-genetic worse
[+420.290ms; +575.710ms] or [+2.780%; +3.809%]
unstable
[-260.191MB; +530.035MB] or [-29.600%; +60.299%]
scenario:renaissance:chi-square worse
[+244.831ms; +1043.169ms] or [+1.543%; +6.574%]
unstable
[-363.536MB; +473.665MB] or [-32.780%; +42.710%]

@pr-commenter
Copy link

pr-commenter bot commented Nov 27, 2025

Benchmarks [aarch64 cpu]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.3 1.35.0-jb_gh_fix_prepare_reports-SNAPSHOT
See matching parameters
Baseline Candidate
alloc off off
cpu on on
iterations 5 5
java "11.0.28" "11.0.28"
memleak off off
modes cpu cpu
wall off off

Summary

Found 0 performance improvements and 4 performance regressions! Performance is the same for 12 metrics, 22 unstable metrics.

scenario Δ mean execution_time Δ mean rss
scenario:renaissance:dotty worse
[+1.208s; +2.860s] or [+2.701%; +6.395%]
unstable
[-114.137MB; +286.824MB] or [-8.137%; +20.447%]
scenario:renaissance:future-genetic worse
[+622.145ms; +781.855ms] or [+4.172%; +5.243%]
unstable
[-248.577MB; +564.147MB] or [-29.155%; +66.168%]
scenario:renaissance:akka-uct worse
[+0.538s; +2.126s] or [+1.819%; +7.195%]
unstable
[-185.314MB; +351.262MB] or [-15.245%; +28.897%]
scenario:renaissance:fj-kmeans worse
[+318.744ms; +1141.256ms] or [+1.507%; +5.396%]
unstable
[-253.223MB; +345.134MB] or [-24.269%; +33.077%]

@pr-commenter
Copy link

pr-commenter bot commented Nov 27, 2025

Benchmarks [x86_64 memleak,alloc]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.3 1.35.0-jb_gh_fix_prepare_reports-SNAPSHOT
See matching parameters
Baseline Candidate
alloc on on
cpu off off
iterations 5 5
java "11.0.28" "11.0.28"
memleak on on
modes memleak,alloc memleak,alloc
wall off off

Summary

Found 0 performance improvements and 2 performance regressions! Performance is the same for 13 metrics, 23 unstable metrics.

scenario Δ mean execution_time Δ mean rss
scenario:renaissance:fj-kmeans worse
[+386.858ms; +501.142ms] or [+1.650%; +2.137%]
unstable
[-240.239MB; +366.618MB] or [-22.799%; +34.792%]
scenario:renaissance:gauss-mix worse
[+704.387ms; +947.613ms] or [+3.888%; +5.230%]
unstable
[-401.632MB; +505.693MB] or [-33.537%; +42.226%]

@pr-commenter
Copy link

pr-commenter bot commented Nov 27, 2025

Benchmarks [aarch64 memleak]

Parameters

Baseline Candidate
config baseline candidate
ddprof 1.34.3 1.35.0-jb_gh_fix_prepare_reports-SNAPSHOT
See matching parameters
Baseline Candidate
alloc off off
cpu off off
iterations 5 5
java "11.0.28" "11.0.28"
memleak on on
modes memleak memleak
wall off off

Summary

Found 0 performance improvements and 5 performance regressions! Performance is the same for 13 metrics, 20 unstable metrics.

scenario Δ mean execution_time Δ mean rss
scenario:renaissance:future-genetic worse
[+362.012ms; +793.988ms] or [+2.408%; +5.282%]
unstable
[-256.348MB; +532.691MB] or [-29.254%; +60.789%]
scenario:renaissance:chi-square worse
[+696.380ms; +1183.620ms] or [+4.416%; +7.506%]
unstable
[-369.391MB; +469.213MB] or [-33.177%; +42.142%]
scenario:renaissance:naive-bayes worse
[+320.838ms; +1203.162ms] or [+2.171%; +8.142%]
unstable
[-287.516MB; +665.318MB] or [-29.927%; +69.252%]
scenario:renaissance:scala-kmeans worse
[+494.235ms; +705.765ms] or [+2.056%; +2.937%]
unstable
[-227.038MB; +338.750MB] or [-23.145%; +34.534%]
scenario:renaissance:philosophers worse
[+415.930ms; +968.070ms] or [+4.178%; +9.723%]
unstable
[-335.023MB; +785.977MB] or [-43.861%; +102.899%]

@jbachorik jbachorik requested review from Copilot and rkennke November 28, 2025 08:04
Copilot finished reviewing on behalf of jbachorik November 28, 2025 08:05
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR attempts to optimize the workflow by adding a failure() condition to the "Prepare reports" steps so that reports are only prepared when they will be uploaded. However, the changes introduce a critical bug that breaks the unwinding reports functionality.

Key Issue

  • The prepare_reports.sh script prepares both test reports (uploaded on failure) and unwinding reports (uploaded on success)
  • Adding failure() to the "Prepare reports" steps prevents unwinding reports from being prepared when tests succeed
  • This breaks the "Upload unwinding reports" step which runs on success() && matrix.config == 'debug'

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jbachorik jbachorik force-pushed the jb/gh_fix_prepare_reports branch from 7450f0e to 347c41b Compare November 28, 2025 08:14
@jbachorik jbachorik changed the title Prepare reports only if and when they are going to be uploaded Always prepare reports Nov 28, 2025
@jbachorik jbachorik force-pushed the jb/gh_fix_prepare_reports branch from 347c41b to 9727b51 Compare November 28, 2025 08:43
@jbachorik jbachorik merged commit 5db4e69 into main Nov 28, 2025
96 checks passed
@jbachorik jbachorik deleted the jb/gh_fix_prepare_reports branch November 28, 2025 09:43
@github-actions github-actions bot added this to the 1.35.0 milestone Nov 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants