Skip to content

Conversation

philprime
Copy link
Member

@philprime philprime commented Aug 27, 2025

📜 Description

Migrates the benchmarking.yml workflow to use dorny/paths-filter for conditional execution on pull requests.

  • A new filter run_benchmarking_for_prs is added to .github/file-filters.yml.
  • The on.pull_request.paths filter is removed from benchmarking.yml.
  • A files-changed job is introduced to detect relevant file changes.
  • The build-benchmark-test-target and run-ui-tests-with-sauce jobs are now conditional based on the files-changed output.
  • A benchmarking-required-check job is added, which always runs to serve as a reliable required check for branch protection.

💡 Motivation and Context

This change is part of a larger effort (see #5951) to migrate all workflows using on.[event].paths filtering to a files-changed job with dorny/paths-filter. This allows for more robust required checks in branch protection rules.

For the benchmarking workflow, this ensures:

  • Benchmarks only run for pull requests when relevant files are changed, optimizing CI resource usage.
  • A consistent benchmarking-required-check is always available for branch protection, regardless of whether the main benchmark jobs are skipped.

This follows the pattern established in #5893 for other workflows.
Fixes #5955

💚 How did you test it?

The changes were verified by observing the CI behavior on this pull request, ensuring the files-changed job correctly identifies relevant changes and the main jobs run or skip as expected, and the benchmarking-required-check job always completes.

📝 Checklist

You have to check all boxes before merging:

  • I added tests to verify the changes.
  • No new PII added or SDK only sends newly added PII if sendDefaultPII is enabled.
  • I updated the docs if needed.
  • I updated the wizard if needed.
  • Review from the native team if needed.
  • No breaking change or entry added to the changelog.
  • No breaking change for hybrid SDKs or communicated to hybrid SDKs.

Open in Cursor

#skip-changelog
[skip ci]

@philprime philprime changed the title Implement required checks for benchmarking chore(ci): Add required checks for benchmarking Aug 27, 2025
@philprime philprime self-assigned this Aug 27, 2025
Copy link

codecov bot commented Aug 27, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.735%. Comparing base (2e79f5f) to head (89f24af).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

Impacted file tree graph

@@              Coverage Diff              @@
##              main     #5996       +/-   ##
=============================================
+ Coverage   86.589%   86.735%   +0.146%     
=============================================
  Files          424       424               
  Lines        36710     36723       +13     
  Branches     15739     17362     +1623     
=============================================
+ Hits         31787     31852       +65     
+ Misses        4881      4824       -57     
- Partials        42        47        +5     

see 20 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2e79f5f...89f24af. Read the comment docs.

@philprime philprime marked this pull request as ready for review August 27, 2025 17:39
cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

Copy link
Contributor

@itaybre itaybre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

cursor[bot]

This comment was marked as outdated.

Copy link
Contributor

github-actions bot commented Aug 28, 2025

Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1208.00 ms 1230.98 ms 22.98 ms
Size 23.75 KiB 928.86 KiB 905.11 KiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
a2a3bfb 1227.94 ms 1261.26 ms 33.32 ms
b9aacb6 1230.42 ms 1251.00 ms 20.58 ms
51b7dd3 1235.06 ms 1258.21 ms 23.15 ms
7d23639 1237.93 ms 1243.04 ms 5.11 ms
c03a8d8 1234.77 ms 1259.19 ms 24.42 ms
42a95d5 1206.00 ms 1224.26 ms 18.26 ms
b57ee62 1218.21 ms 1248.94 ms 30.73 ms
ebc3a34 1236.24 ms 1261.02 ms 24.78 ms
5196f0d 1213.35 ms 1231.37 ms 18.02 ms
ae7be93 1236.24 ms 1258.18 ms 21.94 ms

App size

Revision Plain With Sentry Diff
a2a3bfb 23.75 KiB 872.67 KiB 848.92 KiB
b9aacb6 23.75 KiB 913.64 KiB 889.89 KiB
51b7dd3 23.75 KiB 913.26 KiB 889.52 KiB
7d23639 23.75 KiB 891.01 KiB 867.26 KiB
c03a8d8 23.75 KiB 928.15 KiB 904.40 KiB
42a95d5 23.75 KiB 906.08 KiB 882.33 KiB
b57ee62 23.75 KiB 912.47 KiB 888.72 KiB
ebc3a34 23.75 KiB 919.91 KiB 896.16 KiB
5196f0d 23.75 KiB 876.93 KiB 853.19 KiB
ae7be93 23.75 KiB 879.24 KiB 855.49 KiB

Previous results on branch: cursor/implement-required-checks-for-benchmarking-923d

Startup times

Revision Plain With Sentry Diff
ee602f9 1204.31 ms 1226.85 ms 22.54 ms

App size

Revision Plain With Sentry Diff
ee602f9 23.75 KiB 928.15 KiB 904.40 KiB

@philprime philprime merged commit d157d83 into main Aug 29, 2025
155 of 158 checks passed
@philprime philprime deleted the cursor/implement-required-checks-for-benchmarking-923d branch August 29, 2025 10:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

chore(ci): Add required checks for benchmarking
3 participants