From b31b4cd07f6e0d9578deb6f0afc807ef7a4b92c3 Mon Sep 17 00:00:00 2001 From: Brandon Dahler Date: Fri, 25 Jul 2025 16:16:37 -0400 Subject: [PATCH 1/3] feat(keystore) Define Branch Key Store read consistency semantics --- .../background.md | 78 +++++++++++++++++++ framework/branch-key-store.md | 31 +++++++- 2 files changed, 108 insertions(+), 1 deletion(-) create mode 100644 changes/2025-07-24_define-branch-key-store-read-consistency-semantics/background.md diff --git a/changes/2025-07-24_define-branch-key-store-read-consistency-semantics/background.md b/changes/2025-07-24_define-branch-key-store-read-consistency-semantics/background.md new file mode 100644 index 00000000..4e817287 --- /dev/null +++ b/changes/2025-07-24_define-branch-key-store-read-consistency-semantics/background.md @@ -0,0 +1,78 @@ +[//]: # "Copyright Amazon.com Inc. or its affiliates. All Rights Reserved." +[//]: # "SPDX-License-Identifier: CC-BY-SA-4.0" + +# Define Branch Key Store read consistency semantics + +# Definitions + +## Conventions used in this document + +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", +"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be +interpreted as described in [RFC 2119](https://tools.ietf.org/html/rfc2119). + +# Background + +The [branch key store](../../framework/branch-key-store.md) persists branch key versions +in a persistent data store such as a DynamoDb table. + +These distributed data stores often offer configurable read consistency, +allowing customers to choose eventually consistent reads +for optimal cost and performance +if stale results are acceptable for their use-case. + +The current specification does not specify when agents should expect +the results of the read operations to reflect the updated state caused by mutation operations. + +The current implementation utilizes eventually consistent reads in all current usage. + +Consequently, a read operation will not necessarily reflect the latest results +of a mutation operation +even if performed by the same agent in a serialized manner. + +This change adds the option to require consistent reads and +clarifies the required consistent read behavior for read operations. + +## Detailed Explanation + +The Branch Key Store's `GetActiveBranchKey`, `GetBranchKeyVersion`, and `GetBeaconKey` operations +utilize AWS DDB `GetItem` calls which +do not set the `ConsistentRead` flag. + +When the `ConsistentRead` flag is unset or `false`, +DynamoDb only provides eventually consistent read guarantees. + +This eventually consistent behavior is intentional +in order to optimize performance and cost. + +However, +customers may have non-standard situations +which explicitly require consistent reads to be performed. + +While key management operations SHOULD occur in control plane operations, +those same control plane operations MAY need to perform encryption operations. + +Usage of consistent reads ensures that encryption operations +immediately following the `CreateKey` operation +will be able to find the created keys. + +Similarly, +key rotation and re-encryption operations +performed due to key compromise +MUST use consistent reads to ensure that +the encryption operations utilize the new branch key version. + +# Changes + +The change is to introduce a new, optional `Require Consistent Reads` option +to the Branch Key Store specification. + +This updates: + +- The [Branch Key Store's Initialization](../../framework/branch-key-store.md#initialization) +- The [Branch Key Store's GetActiveBranchKey operation](../../framework/branch-key-store.md#getactivebranchkey) +- The [Branch Key Store's GetBranchKeyVersion operation](../../framework/branch-key-store.md#getbranchkeyversion) +- The [Branch Key Store's GetBeaconKey operation](../../framework/branch-key-store.md#getbeaconkey) + +This update clarifies the existing behavior in a backwards-compatible manner +while allowing agents to opt-in to consistent reads when needed. diff --git a/framework/branch-key-store.md b/framework/branch-key-store.md index 479f086d..cec3e594 100644 --- a/framework/branch-key-store.md +++ b/framework/branch-key-store.md @@ -5,10 +5,12 @@ ## Version -0.9.0 +0.10.0 ### Changelog +- 0.10.0 + - [Define Branch Key Store read consistency semantics](../changes/2025-07-24_define-branch-key-store-read-consistency-semantics/background.md) - 0.9.0 - Re-add [Mitigate Version Race Condition in the Branch Key Store](../changes/2025-01-16_key-store-mitigate-update-race/background.md) with DynamoDB as the only branch key storage option - 0.8.0 @@ -76,6 +78,7 @@ The following inputs MAY be specified to create a KeyStore: - [ID](#keystore-id) - [AWS KMS Grant Tokens](#aws-kms-grant-tokens) - [DynamoDb Client](#dynamodb-client) +- [Require Consistent Reads](#require-consistent-reads) - [KMS Client](#kms-client) The following inputs MUST be specified to create a KeyStore: @@ -112,6 +115,20 @@ and no DynamoDb Client is provided, a new DynamoDb Client MUST be created with the region configured in the MRDiscovery. +### Require Consistent Reads + +Whether read operations are required to perform consistent reads, +ensuring that recent write operations are reflected in their response. + +If set to `true`, +read operations MUST return results +which reflect the result of +the most recently executed write operations. + +If unset or set to `false`, +read operations MAY return stale results +which are caused by eventual consistency. + ### KMS Client The KMS Client used when wrapping and unwrapping keys. @@ -527,6 +544,10 @@ To get the active version for the branch key id from the keystore this operation MUST call AWS DDB `GetItem` using the `branch-key-id` as the Partition Key and `"branch:ACTIVE"` value as the Sort Key. +The [Require Consistent Reads](#require-consistent-reads) configuration MUST be honored +by setting `ConsistentRead` to `true` +when consistent reads are required. + The AWS DDB response MUST contain the fields defined in the [branch keystore record format](#record-format). If the record does not contain the defined fields, this operation MUST fail. @@ -549,6 +570,10 @@ On invocation, the caller: To get a branch key from the keystore this operation MUST call AWS DDB `GetItem` using the `branch-key-id` as the Partition Key and "branch:version:" + `branchKeyVersion` value as the Sort Key. +The [Require Consistent Reads](#require-consistent-reads) configuration MUST be honored +by setting `ConsistentRead` to `true` +when consistent reads are required. + The AWS DDB response MUST contain the fields defined in the [branch keystore record format](#record-format). If the record does not contain the defined fields, this operation MUST fail. @@ -570,6 +595,10 @@ On invocation, the caller: To get a branch key from the keystore this operation MUST call AWS DDB `GetItem` using the `branch-key-id` as the Partition Key and "beacon:ACTIVE" value as the Sort Key. +The [Require Consistent Reads](#require-consistent-reads) configuration MUST be honored +by setting `ConsistentRead` to `true` +when consistent reads are required. + The AWS DDB response MUST contain the fields defined in the [branch keystore record format](#record-format). If the record does not contain the defined fields, this operation MUST fail. From 8b788123dc490f5feb077e16c32b1c4a93d0ca32 Mon Sep 17 00:00:00 2001 From: Brandon Dahler Date: Fri, 25 Jul 2025 16:49:39 -0400 Subject: [PATCH 2/3] Update changes/2025-07-24_define-branch-key-store-read-consistency-semantics/background.md Co-authored-by: seebees --- .../background.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/changes/2025-07-24_define-branch-key-store-read-consistency-semantics/background.md b/changes/2025-07-24_define-branch-key-store-read-consistency-semantics/background.md index 4e817287..15cd09ea 100644 --- a/changes/2025-07-24_define-branch-key-store-read-consistency-semantics/background.md +++ b/changes/2025-07-24_define-branch-key-store-read-consistency-semantics/background.md @@ -59,8 +59,14 @@ will be able to find the created keys. Similarly, key rotation and re-encryption operations performed due to key compromise -MUST use consistent reads to ensure that +MAY want use consistent reads to ensure that the encryption operations utilize the new branch key version. +For this to work local caches would need to be flushed as well. +So this may be prohibitively complicated. + +However, trying to pick and chose which read operations need consistent reads +creates sharp edges in complicated customer systems. +So all operations MUST use the same configuration setting. # Changes From c277763d7b532aef79cd7975c910398893e527f1 Mon Sep 17 00:00:00 2001 From: Ryan Emery Date: Fri, 25 Jul 2025 16:16:46 -0700 Subject: [PATCH 3/3] update some locations --- framework/branch-key-store.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/framework/branch-key-store.md b/framework/branch-key-store.md index cec3e594..60c47e97 100644 --- a/framework/branch-key-store.md +++ b/framework/branch-key-store.md @@ -264,6 +264,7 @@ This MUST include: - [logical Keystore name](#logical-keystore-name) - [AWS KMS Grant Tokens](#aws-kms-grant-tokens) - [AWS KMS Configuration](#aws-kms-configuration) +- [Require Consistent Reads](#require-consistent-reads) ### CreateKeyStore @@ -453,6 +454,10 @@ VersionKey MUST first get the active version for the branch key from the keystor by calling AWS DDB `GetItem` using the `branch-key-id` as the Partition Key and `"branch:ACTIVE"` value as the Sort Key. +The [Require Consistent Reads](#require-consistent-reads) configuration MUST be honored +by setting `ConsistentRead` to `true` +when consistent reads are required. + The `kms-arn` field of DDB response item MUST be [compatible with](#aws-key-arn-compatibility) the configured `KMS ARN` in the [AWS KMS Configuration](#aws-kms-configuration) for this keystore.