Skip to content

ADR-53: JetStream Read-after-Write #358

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft

ADR-53: JetStream Read-after-Write #358

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b91f42b
ADR-53: JetStream Read-after-Write
MauriceVanVeen Jul 11, 2025
f0aabb4
ADR-53: Client implementation
MauriceVanVeen Jul 31, 2025
dcb9074
ADR-53: Implemented in 2.12
MauriceVanVeen Aug 1, 2025
21d5059
ADR-53: kv.Create's kv.Get requires a min last sequence
MauriceVanVeen Aug 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ This repository captures Architecture, Design Specifications and Feature Guidanc
|[ADR-49](adr/ADR-49.md)|jetstream, spec, 2.12|JetStream Distributed Counter CRDT|
|[ADR-50](adr/ADR-50.md)|jetstream, server, client, 2.12|JetStream Batch Publishing|
|[ADR-52](adr/ADR-52.md)|jetstream, client, refinement, 2.12|No Headers support for Direct Get (updating [ADR-31](adr/ADR-31.md))|
|[ADR-53](adr/ADR-53.md)|jetstream, kv, objectstore, server, client, refinement, 2.12|JetStream Read-after-Write (updating [ADR-8](adr/ADR-8.md), [ADR-17](adr/ADR-17.md), [ADR-20](adr/ADR-20.md), [ADR-31](adr/ADR-31.md), [ADR-37](adr/ADR-37.md))|

## Client

Expand Down Expand Up @@ -56,6 +57,7 @@ This repository captures Architecture, Design Specifications and Feature Guidanc
|[ADR-48](adr/ADR-48.md)|jetstream, client, kv, refinement, 2.11|TTL Support for Key-Value Buckets (updating [ADR-8](adr/ADR-8.md))|
|[ADR-50](adr/ADR-50.md)|jetstream, server, client, 2.12|JetStream Batch Publishing|
|[ADR-52](adr/ADR-52.md)|jetstream, client, refinement, 2.12|No Headers support for Direct Get (updating [ADR-31](adr/ADR-31.md))|
|[ADR-53](adr/ADR-53.md)|jetstream, kv, objectstore, server, client, refinement, 2.12|JetStream Read-after-Write (updating [ADR-8](adr/ADR-8.md), [ADR-17](adr/ADR-17.md), [ADR-20](adr/ADR-20.md), [ADR-31](adr/ADR-31.md), [ADR-37](adr/ADR-37.md))|

## Jetstream

Expand Down Expand Up @@ -87,6 +89,7 @@ This repository captures Architecture, Design Specifications and Feature Guidanc
|[ADR-49](adr/ADR-49.md)|jetstream, spec, 2.12|JetStream Distributed Counter CRDT|
|[ADR-50](adr/ADR-50.md)|jetstream, server, client, 2.12|JetStream Batch Publishing|
|[ADR-52](adr/ADR-52.md)|jetstream, client, refinement, 2.12|No Headers support for Direct Get (updating [ADR-31](adr/ADR-31.md))|
|[ADR-53](adr/ADR-53.md)|jetstream, kv, objectstore, server, client, refinement, 2.12|JetStream Read-after-Write (updating [ADR-8](adr/ADR-8.md), [ADR-17](adr/ADR-17.md), [ADR-20](adr/ADR-20.md), [ADR-31](adr/ADR-31.md), [ADR-37](adr/ADR-37.md))|

## Kv

Expand All @@ -95,13 +98,15 @@ This repository captures Architecture, Design Specifications and Feature Guidanc
|[ADR-8](adr/ADR-8.md)|jetstream, client, kv, spec|JetStream based Key-Value Stores|
|[ADR-19](adr/ADR-19.md)|jetstream, client, kv, objectstore|API prefixes for materialized JetStream views|
|[ADR-48](adr/ADR-48.md)|jetstream, client, kv, refinement, 2.11|TTL Support for Key-Value Buckets (updating [ADR-8](adr/ADR-8.md))|
|[ADR-53](adr/ADR-53.md)|jetstream, kv, objectstore, server, client, refinement, 2.12|JetStream Read-after-Write (updating [ADR-8](adr/ADR-8.md), [ADR-17](adr/ADR-17.md), [ADR-20](adr/ADR-20.md), [ADR-31](adr/ADR-31.md), [ADR-37](adr/ADR-37.md))|

## Objectstore

|Index|Tags|Description|
|-----|----|-----------|
|[ADR-19](adr/ADR-19.md)|jetstream, client, kv, objectstore|API prefixes for materialized JetStream views|
|[ADR-20](adr/ADR-20.md)|jetstream, client, objectstore, spec|JetStream based Object Stores|
|[ADR-53](adr/ADR-53.md)|jetstream, kv, objectstore, server, client, refinement, 2.12|JetStream Read-after-Write (updating [ADR-8](adr/ADR-8.md), [ADR-17](adr/ADR-17.md), [ADR-20](adr/ADR-20.md), [ADR-31](adr/ADR-31.md), [ADR-37](adr/ADR-37.md))|

## Observability

Expand All @@ -122,6 +127,7 @@ This repository captures Architecture, Design Specifications and Feature Guidanc
|-----|----|-----------|
|[ADR-48](adr/ADR-48.md)|jetstream, client, kv, refinement, 2.11|TTL Support for Key-Value Buckets (updating [ADR-8](adr/ADR-8.md))|
|[ADR-52](adr/ADR-52.md)|jetstream, client, refinement, 2.12|No Headers support for Direct Get (updating [ADR-31](adr/ADR-31.md))|
|[ADR-53](adr/ADR-53.md)|jetstream, kv, objectstore, server, client, refinement, 2.12|JetStream Read-after-Write (updating [ADR-8](adr/ADR-8.md), [ADR-17](adr/ADR-17.md), [ADR-20](adr/ADR-20.md), [ADR-31](adr/ADR-31.md), [ADR-37](adr/ADR-37.md))|

## Security

Expand Down Expand Up @@ -160,6 +166,7 @@ This repository captures Architecture, Design Specifications and Feature Guidanc
|[ADR-43](adr/ADR-43.md)|jetstream, client, server, 2.11|JetStream Per-Message TTL|
|[ADR-44](adr/ADR-44.md)|jetstream, server, 2.11|Versioning for JetStream Assets|
|[ADR-50](adr/ADR-50.md)|jetstream, server, client, 2.12|JetStream Batch Publishing|
|[ADR-53](adr/ADR-53.md)|jetstream, kv, objectstore, server, client, refinement, 2.12|JetStream Read-after-Write (updating [ADR-8](adr/ADR-8.md), [ADR-17](adr/ADR-17.md), [ADR-20](adr/ADR-20.md), [ADR-31](adr/ADR-31.md), [ADR-37](adr/ADR-37.md))|

## Spec

Expand Down
4 changes: 0 additions & 4 deletions adr-template.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,6 @@

[If this is a specification or actual design, write something here.]

## Decision

[Maybe this was just an architectural decision...]

## Consequences

[Any consequences of this design, such as breaking change or Vorpal Bunnies]
33 changes: 20 additions & 13 deletions adr/ADR-31.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,12 @@
| Status | Implemented |
| Tags | jetstream, client, server, 2.11 |

| Revision | Date | Author | Info |
|----------|------------|------------|----------------------------------------------------------|
| 1 | 2022-08-08 | @tbeets | Initial design |
| 2 | 2024-03-06 | @ripienaar | Adds Multi and Batch behaviors for Server 2.11 |
| 3 | 2025-06-19 | @ripienaar | Support surpressing headers in replies using `NoHeaders` |
| Revision | Date | Author | Info | Refinement | Server Requirement |
|----------|------------|-----------------|----------------------------------------------------------|------------|--------------------|
| 1 | 2022-08-08 | @tbeets | Initial design | | |
| 2 | 2024-03-06 | @ripienaar | Adds Multi and Batch behaviors for Server 2.11 | | |
| 3 | 2025-06-19 | @ripienaar | Support suppressing headers in replies using `NoHeaders` | | |
| 4 | 2025-07-11 | @MauriceVanVeen | Update on Read-after-Write guarantee | ADR-53 | |

## Context and motivation

Expand Down Expand Up @@ -42,14 +43,20 @@ clients. Also, read availability can be enhanced as mirrors may be available to

###### A note on read-after-write coherency

The existing Get API `$JS.API.STREAM.MSG.GET.<stream>` provides read-after-write coherency by routing requests to a
stream's current peer leader (R>1) or single server (R=1). A client that publishes a message to stream (with ACK) is
assured that a subsequent call to the Get API will return that message as the read will go a server that defines
_most current_.
The existing Get API `$JS.API.STREAM.MSG.GET.<stream>` as well as _Direct Get_ do NOT provide any read-after-write
guarantees by default. The existing Get API only guarantees read-after-write if the underlying stream is not
replicated (R=1).

In contrast, _Direct Get_ does not assure read-after-write coherency as responders may be non-leader stream servers
(that may not have yet applied the latest consensus writes) or MIRROR downstream servers that have not yet _consumed_
the latest consensus writes from upstream.
_Direct Get_ does not assure read-after-write coherency as responders may be non-leader stream servers (that may not
have yet applied the latest consensus writes) or MIRROR downstream servers that have not yet _consumed_ the latest
consensus writes from upstream.

The Get API routes requests to a stream's current peer leader (R>1). A client that publishes multiple messages to a
stream (with ACK) is assured that they will be properly ordered by sequence, regardless of which peer leader was active
at that time. However, during and after leader elections, calls to the Get API could still be served by a server that
still thinks it's leader even if a new leader was elected in the meantime (but it doesn't know yet).

Read-after-write guarantees can be opted into with [ADR-53](adr/ADR-53.md).

## Implementation

Expand All @@ -61,7 +68,7 @@ the latest consensus writes from upstream.
based on `max_msgs_per_subject`

> Allow Direct is set automatically based on the inferred use case of the stream. Maximum messages per subject is a
tell-tale of a stream that is a KV bucket.
> tell-tale of a stream that is a KV bucket.

### Direct Get API

Expand Down
Loading