Skip to content

Conversation

DAlperin
Copy link
Member

The limits nightly test seems to exhibit a pathological case wherein there are no updates, but the frontier moves forward frequently leading to lots of empty batches. In theory we should handle this fine, but we don't (I'm actively investigating this). Anyways, this seems like a good balance to strike between the old compaction behavior and incremental compaction, while avoiding some of the empty batch downsides.

Motivation

Tips for reviewer

Checklist

  • This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
  • This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
  • If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
  • If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
  • If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.

The limits nightly test seems to exhibit a pathological case wherein 
there are no updates, but the frontier moves forward frequently leading
to _lots_ of empty batches. In theory we should handle this fine, but we
don't (I'm actively investigating this). Anyways, this seems like a good
balance to strike between the old compaction behavior and incremental
compaction, while avoiding some of the empty batch downsides.
@DAlperin DAlperin requested a review from a team as a code owner September 22, 2025 04:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant