feat: add retries to sql macros #2529

MasterPtato · 2025-06-03T20:19:45Z

Changes

MasterPtato · 2025-06-03T20:20:25Z

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

How to use the Graphite Merge Queue

Add the label merge-queue to this PR to add it to the merge queue.

You must have a Graphite account in order to use the merge queue. Sign up using this link.

_{An organization admin has enabled the Graphite Merge Queue in this repository.} _{Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.}

This stack of pull requests is managed by Graphite. Learn more about stacking.

greptile-apps

PR Summary

Adds SQL query retry mechanism with exponential backoff and improves memory efficiency across the codebase.

Added retry logic in sql_query_macros.rs with 100ms base retry and max 16 retries
Removed MaxSqlRetries error variant from WorkflowError in favor of macro-level retry handling
Renamed query to txn in database drivers for better semantic clarity
Added reference operators (&) to SQL query parameters across multiple services to prevent unnecessary cloning
Simplified transaction retry logic to use consistent backoff for all database errors

_{12 file(s) reviewed, 3 comment(s)}
_{Edit PR Review Bot Settings | Greptile}

greptile-apps · 2025-06-03T20:21:00Z

packages/common/chirp-workflow/core/src/db/fdb_sqlite_nats/mod.rs

 							if is_retrying.load(Ordering::Relaxed) {
-								self.query(|| async {
-									sql_execute!(
-										[self, &pool]
-										"
-										DELETE FROM workflow_signal_events
-										WHERE location = jsonb(?1)
-										",
-										location,
-									)
-									.await
-								})
+								sql_execute!(
+									[self, &pool]
+									"
+									DELETE FROM workflow_signal_events
+									WHERE location = jsonb(?1)
+									",
+									location,
+								)
 								.await
 								.map_err(|x| fdb::FdbBindingError::CustomError(x.into()))?;
 							}


logic: Potential race condition: the DELETE operation should be in the same transaction as the INSERT to maintain atomicity

greptile-apps · 2025-06-03T20:21:34Z

packages/common/pools/src/utils/sql_query_macros.rs

+			let mut backoff = $crate::__rivet_util::Backoff::new(
+				4,
+				None,
+				$crate::utils::sql_query_macros::QUERY_RETRY_MS,
+				50
+			);


logic: Backoff parameters are inconsistent - base=4 and factor=50 here vs QUERY_RETRY_MS=100 defined at the top. Consider aligning these values or documenting why they differ.

greptile-apps · 2025-06-03T20:21:35Z

packages/common/pools/src/utils/sql_query_macros.rs

+							Database(_) | Io(_) | Tls(_) | Protocol(_) | PoolTimedOut | PoolClosed
+							| WorkerCrashed => {


style: Consider adding RowNotFound to non-retryable errors since retrying won't help if the row doesn't exist

Suggested change

Database(_) | Io(_) | Tls(_) | Protocol(_) | PoolTimedOut | PoolClosed

| WorkerCrashed => {

Database(_) | Io(_) | Tls(_) | Protocol(_) | PoolTimedOut | PoolClosed

| WorkerCrashed | RowNotFound => {

cloudflare-workers-and-pages · 2025-06-03T20:23:18Z

Deploying rivet with Cloudflare Pages

Latest commit:	`c307626`
Status:	✅ Deploy successful!
Preview URL:	https://758a1d78.rivet.pages.dev
Branch Preview URL:	https://06-03-feat-add-retries-to-sq.rivet.pages.dev

View logs

cloudflare-workers-and-pages · 2025-06-04T00:37:55Z

Deploying rivet-studio with Cloudflare Pages

Latest commit:	`c307626`
Status:	✅ Deploy successful!
Preview URL:	https://bb342d1e.rivet-studio.pages.dev
Branch Preview URL:	https://06-03-feat-add-retries-to-sq.rivet-studio.pages.dev

View logs

cloudflare-workers-and-pages · 2025-06-04T00:38:18Z

Deploying rivet-hub with Cloudflare Pages

Latest commit:	`c307626`
Status:	✅ Deploy successful!
Preview URL:	https://6498ef2a.rivet-hub-7jb.pages.dev
Branch Preview URL:	https://06-03-feat-add-retries-to-sq.rivet-hub-7jb.pages.dev

View logs

graphite-app · 2025-06-09T16:46:57Z

Merge activity

Jun 9, 4:46 PM UTC: MasterPtato added this pull request to the Graphite merge queue.
Jun 9, 4:48 PM UTC: CI is running for this pull request on a draft pull request (#2574) due to your merge queue CI optimization settings.
Jun 9, 4:49 PM UTC: Merged by the Graphite merge queue via draft PR: #2574.

## Changes

MasterPtato requested a review from NathanFlurry June 3, 2025 20:19

greptile-apps bot reviewed Jun 3, 2025

View reviewed changes

MasterPtato force-pushed the 06-03-feat_add_retries_to_sql_macros branch from 200c987 to b3eae4a Compare June 4, 2025 00:34

MasterPtato mentioned this pull request Jun 4, 2025

fix: fix build cache key #2530

Closed

MasterPtato force-pushed the 06-03-fix_add_create_ts_to_server_list branch from 5e33d41 to 7666132 Compare June 4, 2025 19:17

MasterPtato force-pushed the 06-03-feat_add_retries_to_sql_macros branch from b3eae4a to 4fc5f51 Compare June 4, 2025 19:17

This was referenced Jun 4, 2025

feat(clusters): add margin per pool #2543

Closed

fix(pegboard): increase actor log ttl to 14 days #2544

Closed

NathanFlurry force-pushed the 06-03-feat_add_retries_to_sql_macros branch from 4fc5f51 to 406d6f9 Compare June 5, 2025 00:58

NathanFlurry force-pushed the 06-03-fix_add_create_ts_to_server_list branch from 7666132 to 20eace2 Compare June 5, 2025 00:58

MasterPtato force-pushed the 06-03-fix_add_create_ts_to_server_list branch from 20eace2 to b77a826 Compare June 5, 2025 02:10

MasterPtato force-pushed the 06-03-feat_add_retries_to_sql_macros branch from 406d6f9 to d44c0b9 Compare June 5, 2025 02:10

This was referenced Jun 5, 2025

chore: fix dc wf #2552

Closed

fix: add vector http config to edge configs #2553

Closed

feat: add retries to sql macros

c307626

MasterPtato force-pushed the 06-03-feat_add_retries_to_sql_macros branch from d44c0b9 to c307626 Compare June 6, 2025 01:36

MasterPtato force-pushed the 06-03-fix_add_create_ts_to_server_list branch from b77a826 to 863f139 Compare June 6, 2025 01:36

graphite-app bot pushed a commit that referenced this pull request Jun 9, 2025

feat: add retries to sql macros (#2529)

1f680de

 ## Changes

graphite-app bot closed this Jun 9, 2025

graphite-app bot deleted the 06-03-feat_add_retries_to_sql_macros branch June 9, 2025 16:49

github-actions bot mentioned this pull request Jun 9, 2025

chore(main): release 25.5.1 #2423

Merged

MasterPtato mentioned this pull request Jun 9, 2025

fix(pegboard): fix container runner orphaning #2575

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: add retries to sql macros #2529

feat: add retries to sql macros #2529

Uh oh!

MasterPtato commented Jun 3, 2025

Uh oh!

MasterPtato commented Jun 3, 2025 •

edited

Loading

Uh oh!

greptile-apps bot left a comment

Uh oh!

greptile-apps bot Jun 3, 2025

Uh oh!

greptile-apps bot Jun 3, 2025

Uh oh!

greptile-apps bot Jun 3, 2025

Uh oh!

cloudflare-workers-and-pages bot commented Jun 3, 2025 •

edited

Loading

Uh oh!

cloudflare-workers-and-pages bot commented Jun 4, 2025 •

edited

Loading

Uh oh!

cloudflare-workers-and-pages bot commented Jun 4, 2025 •

edited

Loading

Uh oh!

graphite-app bot commented Jun 9, 2025 •

edited

Loading

Uh oh!

Uh oh!

		Database(_) \| Io(_) \| Tls(_) \| Protocol(_) \| PoolTimedOut \| PoolClosed
		\| WorkerCrashed => {

feat: add retries to sql macros #2529

feat: add retries to sql macros #2529

Uh oh!

Conversation

MasterPtato commented Jun 3, 2025

Changes

Uh oh!

MasterPtato commented Jun 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

How to use the Graphite Merge Queue

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

PR Summary

Uh oh!

greptile-apps bot Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

cloudflare-workers-and-pages bot commented Jun 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploying rivet with Cloudflare Pages

Uh oh!

cloudflare-workers-and-pages bot commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploying rivet-studio with Cloudflare Pages

Uh oh!

cloudflare-workers-and-pages bot commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploying rivet-hub with Cloudflare Pages

Uh oh!

graphite-app bot commented Jun 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge activity

Uh oh!

Uh oh!

MasterPtato commented Jun 3, 2025 •

edited

Loading

cloudflare-workers-and-pages bot commented Jun 3, 2025 •

edited

Loading

cloudflare-workers-and-pages bot commented Jun 4, 2025 •

edited

Loading

cloudflare-workers-and-pages bot commented Jun 4, 2025 •

edited

Loading

graphite-app bot commented Jun 9, 2025 •

edited

Loading