Skip to content

feat: dependency job debouncing #6769

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 91 commits into from
Oct 16, 2025
Merged

feat: dependency job debouncing #6769

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
ab61a11
v0
pyranota Sep 9, 2025
48ee617
optimize relocks
pyranota Sep 10, 2025
0e35470
make it work with relative relative imports
pyranota Sep 10, 2025
8ab9a27
use fallback
pyranota Sep 11, 2025
e57f4d8
remove dbg and todos
pyranota Sep 11, 2025
3203162
future proof a bit
pyranota Sep 11, 2025
af5c6a7
cleanup
pyranota Sep 11, 2025
e6b9490
more cleanup
pyranota Sep 11, 2025
ff511bd
remove final TODO
pyranota Sep 11, 2025
5f37946
Merge branch 'main' into fix-relative-imports-cache-invalidation-apps
pyranota Sep 11, 2025
e1013b8
do not use bytemuck
pyranota Sep 11, 2025
5210446
Merge branch 'fix-relative-imports-cache-invalidation-apps' of github…
pyranota Sep 11, 2025
13a5122
optimize hashing
pyranota Sep 11, 2025
fdd24e6
implementation 1
pyranota Sep 11, 2025
0e21fd7
almost v0
pyranota Sep 12, 2025
ece7aac
v0
pyranota Sep 12, 2025
be6e82e
Merge branch 'main' into pyra/win-1566-fix-flowapps-renaming-not-bein…
pyranota Sep 12, 2025
d933e13
add comments and use fallback
pyranota Sep 12, 2025
68ec4f1
call dissolve for apps
pyranota Sep 12, 2025
8766c9f
add comms
pyranota Sep 12, 2025
71da9b7
refactor v0 (partially tested + dirty)
pyranota Sep 16, 2025
90af8aa
finishing
pyranota Sep 16, 2025
111b6cd
Merge branch 'main' into pyra/win-1566-fix-flowapps-renaming-not-bein…
pyranota Sep 16, 2025
7d56970
remove TODO
pyranota Sep 16, 2025
6b98b5f
Update SQLx metadata
actions-user Sep 16, 2025
fb722ac
silence unused argument
pyranota Sep 16, 2025
1da61ab
Merge branch 'pyra/win-1566-fix-flowapps-renaming-not-being-reflected…
pyranota Sep 16, 2025
8ff382c
cleanup
pyranota Sep 16, 2025
a731f09
implement rebuild_map endpoint
pyranota Sep 19, 2025
04d5499
update windmill api client
pyranota Sep 19, 2025
74151de
almost finish with tests
pyranota Sep 19, 2025
5f459ee
add proper testing
pyranota Sep 23, 2025
f22a27c
remove unused fixtures
pyranota Sep 23, 2025
7adedc5
Update SQLx metadata
windmill-internal-app[bot] Sep 23, 2025
1d05166
partial cleanup
pyranota Sep 23, 2025
51e8a7c
Merge branch 'pyra/win-1566-fix-flowapps-renaming-not-being-reflected…
pyranota Sep 23, 2025
0badcf7
Update backend/windmill-worker/src/scoped_dependency_map.rs
pyranota Sep 23, 2025
e3705b2
Update backend/windmill-common/src/scripts.rs
pyranota Sep 23, 2025
74c612f
evil doings
pyranota Sep 23, 2025
c97f5cd
Merge branch 'main' into pyra/win-1566-fix-flowapps-renaming-not-bein…
pyranota Sep 23, 2025
6e7482e
more cleanup
pyranota Sep 23, 2025
fd614f6
Update SQLx metadata
windmill-internal-app[bot] Sep 23, 2025
c9a715d
Merge branch 'main' into pyra/win-1566-fix-flowapps-renaming-not-bein…
pyranota Sep 23, 2025
b5d5ead
more cleanup
pyranota Sep 24, 2025
da2fa42
fixing CI
pyranota Sep 24, 2025
2a08405
remove python from default features
pyranota Sep 24, 2025
f7f12c9
feat: dependency job debouncing
pyranota Oct 7, 2025
1dae459
Merge branch 'main' into djobs-debouncing
pyranota Oct 7, 2025
83ff6b7
Merge branch 'main' into djobs-debouncing
pyranota Oct 8, 2025
c571714
checkpoint
pyranota Oct 8, 2025
c3c7404
Merge branch 'djobs-debouncing' of github.com:windmill-labs/windmill …
pyranota Oct 8, 2025
ad3abde
more improvements
pyranota Oct 9, 2025
782de71
refactor: clean up dependency job debouncing implementation
claude[bot] Oct 9, 2025
5cead55
cleanup
pyranota Oct 9, 2025
390dfe6
Merge branch 'main' into djobs-debouncing
pyranota Oct 9, 2025
ca43cdf
Update SQLx metadata
windmill-internal-app[bot] Oct 9, 2025
bdffd15
Update backend/windmill-common/src/jobs.rs
pyranota Oct 9, 2025
e71838e
Update backend/windmill-common/src/jobs.rs
pyranota Oct 9, 2025
2e39de0
Update backend/windmill-queue/src/jobs.rs
pyranota Oct 9, 2025
b243053
better error handling for helper
pyranota Oct 9, 2025
60bfa61
Merge branch 'djobs-debouncing' of github.com:windmill-labs/windmill …
pyranota Oct 9, 2025
4fd3179
update ee-ref
pyranota Oct 9, 2025
5f62963
test: add race condition test for dependency job debouncing
claude[bot] Oct 9, 2025
69bfd89
test: implement race condition test for dependency job debouncing
claude[bot] Oct 9, 2025
5451d66
implement fallback
pyranota Oct 9, 2025
d49fe34
Merge branch 'djobs-debouncing' of github.com:windmill-labs/windmill …
pyranota Oct 9, 2025
918dfbf
make it mostly work
pyranota Oct 13, 2025
4d180bb
all tests are almost working
pyranota Oct 14, 2025
26abd52
add comments a bit
pyranota Oct 14, 2025
8040edc
Update backend/windmill-common/src/scripts.rs
pyranota Oct 14, 2025
c840fb4
Update backend/windmill-worker/src/worker_lockfiles.rs
pyranota Oct 14, 2025
31f39e2
Update backend/windmill-worker/src/worker_lockfiles.rs
pyranota Oct 14, 2025
4f96460
feat: improve debouncing documentation, tracing, and error handling
github-actions[bot] Oct 14, 2025
c828cc2
Merge branch 'main' into djobs-debouncing
pyranota Oct 15, 2025
23ce8fb
do some work for future improvements
pyranota Oct 15, 2025
6b1dd46
fix tests
pyranota Oct 15, 2025
53e3ba1
clippy
pyranota Oct 15, 2025
618c44b
update sqlx
pyranota Oct 15, 2025
8d88293
clippy
pyranota Oct 15, 2025
ab048b7
update ee ref
pyranota Oct 15, 2025
4e50d9d
flag tests behind the feature, add timeout
pyranota Oct 16, 2025
62c7753
fix timeout + cleanup
pyranota Oct 16, 2025
8dd5e78
cleanup
pyranota Oct 16, 2025
e4b31a8
row lock debounce_key
pyranota Oct 16, 2025
967c2c1
addressing TODOs
pyranota Oct 16, 2025
78aa934
fix test
pyranota Oct 16, 2025
94b1129
Merge branch 'main' into djobs-debouncing
pyranota Oct 16, 2025
d9aa6af
ee ref
pyranota Oct 16, 2025
0460633
Merge branch 'main' into djobs-debouncing
pyranota Oct 16, 2025
233db2e
Merge branch 'main' into djobs-debouncing
rubenfiszel Oct 16, 2025
de3b0dc
ee repo
pyranota Oct 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions backend/.sqlx/query-04ce5c530c80ae6f911dfe0dc9ed7d1a2e10342bbbc7f8486df0b73f5657a493.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions backend/.sqlx/query-08e5832c4002a0970d4105bb80371dcc58cb037a59afe08cd1372b51791b4165.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions backend/.sqlx/query-1c047fef05e8cfc07aef7ea9a5454334eb7375b483e8ffd80364c60c3aad04b4.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions backend/.sqlx/query-1c739fddea33331fdb5acbfe614dc18a4db630616b3aa80c27b54e4bb4e20f30.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions backend/.sqlx/query-3528fbb71569b9bfd2a66d0692f9f0d1ecabb750d3d508f5c6a5cb3eaf3cd209.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions backend/.sqlx/query-3af3c1080cde18cfbeb08e290f63b465b4bbf83bc86c0efacd6133bb432ee13d.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions backend/.sqlx/query-421c7e0388326889b33fae4c8fa2fd7cb4a7be6dc50c9f7507eaa15f138484dc.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions backend/.sqlx/query-44d3e7dce67967471fa638f2beedf8567579218d5c4332b01e169732c836ebdc.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions backend/.sqlx/query-47a55edc0f5c54ac9f5e48665c8dafc044d1861048857ec2fc70f929ab358373.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions backend/.sqlx/query-53b236c65e8790b7474e919daa0583902596c4faed6d83e668be85aba8eb644a.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions backend/.sqlx/query-5848d416e2d96eb20c7417dec608f8ca36db091c67cf9ec5e7cec40486532ab9.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions backend/.sqlx/query-5bedbfe98b3cacd1c6f4b2344a9a626245abe58c306392a76710252d16d0bd44.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions backend/.sqlx/query-6641d63a691d712bc24be9c972646fed29a76e579e72d6c8539cd0de73b424db.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 2 additions & 3 deletions ...438c9855ba860e23befd88c2544ef86e9133.json → ...73fc77421710992b10f526aa36bc64ff0930.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions backend/.sqlx/query-720d7f0d258d52a6b89e2f4b32ea0fd360da8a9f06e0ab59d9e724851ced4247.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

22 changes: 22 additions & 0 deletions backend/.sqlx/query-73fcf81d272c1613e094d60c0a088f9d694bc37caacef7269e3738de8b5f6013.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 14 additions & 0 deletions backend/.sqlx/query-76774e6f72c8c8b7473487e4176dc17b17372b7292e39d3888a93ff4fe49e4f5.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions backend/.sqlx/query-77f33de1d95e38a44968ea7f026476226c66fe42c9f992bd4c6c232a68ade2bd.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 4 additions & 4 deletions ...c0c60d4ec155f79d067a8b74cf24eebad88c.json → ...f5d7df9acf74de6dcc566924de02f1807af2.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading