StatefulSet Support for Updating Volume Claim Template #1

huww98 · 2024-05-18T03:06:11Z

One-line PR description:

Issue link:

Other comments:

vie-serendipity · 2024-05-21T02:54:53Z

keps/sig-storage/NNNN-stateful-set-update-claim-template/README.md

+   e.g.:
+   - If `spec.updateStrategy.type` is `RollingUpdate`,
+     update the PVCs in the order from the largest ordinal to the smallest.
+     Only proceed to the next ordinal when all the PVCs of the previous ordinal


Does just being compatible mean we can move on to the next ordinal? Shouldn't the resizing be finished?

By saying compatible, we are already waiting for the resize to finish. Because in "What PVC is compatible", status.capacity.storage should be updated before a PVC becomes compatible.

vie-serendipity · 2024-05-21T03:12:09Z

keps/sig-storage/NNNN-stateful-set-update-claim-template/README.md

+- totalCapacity: the sum of `status.capacity` of all the PVCs.
+
+Some fields in the `status` are also updated to reflect the staus of the PVCs:
+- readyReplicas: in addition to pods, also consider the PVCs status. A PVC is not ready if:


If strategy is ondelete, and old pvc works, the corresponding pod is ready or not?

if volumeClaimUpdateStrategy is OnDelete, the PVC is always considered ready. This is required to not break the current behavior.

vie-serendipity · 2024-05-21T03:15:25Z

keps/sig-storage/NNNN-stateful-set-update-claim-template/README.md

+   (`spec.resources.requests.storage`, `spec.volumeAttributesClassName`, `metadata.labels`, and `metadata.annotations` currently),
+   update the PVC in-place to the extent possible.
+   Do not perform the update that will be rejected by API server, such as
+   decreasing the storage size below its current status.


Change API server to allow any updates to volumeClaimTemplate of a StatefulSet.

I'm confused. Will decreasing the storage size be supported in this proposal?

Decreasing in template is supported, but only for the purpose of recovering from failed expansion, or as template for new PVCs. Shrink the PV/PVC is not supported.

We accept any update to the template, but decreasing the size will not affect the existing healthy PVC.

vie-serendipity · 2024-05-22T13:06:44Z

keps/sig-storage/4650-stateful-set-update-claim-template/README.md

+  - `volumeClaimUpdateStrategy` is `InPlace` and the PVC is updating;
+- availableReplicas: total number of replicas of which both Pod and PVCs are ready for at least `minReadySeconds`
+- currentRevision, updateRevision, currentReplicas, updatedReplicas
+  are updated to reflect the status of PVCs.


When expanding the size of pvcs and then kubectl rollout undo, actually pvcs will not be rollbacked, they will stay the expanded size. Right?

It depends. If you enabled feature gate RecoverVolumeExpansionFailure, and the PVC pvc.Status.AllocatedResources is lower, then PVC is rolled back, and expansion is cancelled.

This is what I mean by writing "update the PVC in-place to the extent possible". Maybe I should make this more explicit.

vie-serendipity · 2024-05-22T13:12:57Z

keps/sig-storage/4650-stateful-set-update-claim-template/README.md

+
+#### Story 5: Asymmetric Replicas
+
+The replicas of our StatefulSet are not identical, so we still want to update


Suggested change

The replicas of our StatefulSet are not identical, so we still want to update

The templates for different replicas of a StatefulSet are not identical, so we still want to update

I would like the word "template" to specifically refer to the one written in StatefulSet. So I don't like "templates for different replicas". Maybe "The storage requirement of different replicas"?

that's fine.

vie-serendipity · 2024-05-22T13:18:09Z

keps/sig-storage/4650-stateful-set-update-claim-template/README.md

+
+We're running a CI/CD system and the end-to-end automation is desired.
+To expand the volumes managed by a StatefulSet,
+we can just use the same pipeline that we are already using to updating the Pod.


Suggested change

we can just use the same pipeline that we are already using to updating the Pod.

we can just use the same pipeline that we are already using to update the Pod.

vie-serendipity · 2024-05-22T13:25:24Z

keps/sig-storage/4650-stateful-set-update-claim-template/README.md

+2. Introduce a new field in StatefulSet `spec.updateStrategy.rollingUpdate`: `volumeClaimSyncStrategy`
+   to specify how to update PVCs and Pods. Possible values are:
+   - `Async`: the default value, preseve the current behavior.
+   - `LockStep`: update PVCs first, then update Pods. See below for details.


I don't understand why it's called LockStep. I came up with a few names for you to consider. Stepwise, Serial, Ordered, Sync.

kubernetes#3412 (comment)

See here for the context of word lock-step.

…plate"

Production Readiness review, etc.

…lds.

Pod may be deleted externally, e.g. evicted.

huww98 force-pushed the sts-update-claim branch 2 times, most recently from a40a3f1 to a5e98fa Compare May 19, 2024 11:21

vie-serendipity reviewed May 22, 2024

View reviewed changes

vie-serendipity suggested changes May 22, 2024

View reviewed changes

huww98 force-pushed the sts-update-claim branch from 294de45 to 9ef734c Compare May 27, 2024 02:48

huww98 added 18 commits May 25, 2025 00:51

initial version of "StatefulSet Support for Updating Volume Claim Tem…

1303775

…plate"

what is capatible

5533b98

volumeClaimSyncStrategy

dd41fed

User stories

3334f65

we are already waiting for PVC deletion

17f9f77

status

7b86724

check compatible before advancing updatedReplicas

d1fb4ed

add Kubernetes API Changes section

b73739d

KEP-4650

f303607

rewrite "When to update PVCs" and other updates

fbe0992

volumeClaimTemplates

175b7d0

Change the owning-sig to sig-apps

e88af87

update for comments

cfa7473

Production Readiness review, etc.

some clarifications

82d0a10

Remove volumeClaimSyncStrategy. Don't allow editing immutable PVC fie…

7b991cb

…lds.

update with the implementation

d35c83a

volumeClaimUpdate{Strategy => Policy}

e9bef51

merge template update

046c771

huww98 force-pushed the sts-update-claim branch 2 times, most recently from 95ef855 to 3cb46c6 Compare May 25, 2025 11:12

PVC compatible => ready

bad859d

huww98 force-pushed the sts-update-claim branch from 3cb46c6 to bad859d Compare May 25, 2025 16:21

huww98 force-pushed the sts-update-claim branch from d274fd7 to 937c421 Compare June 10, 2025 16:27

address comments

7992ee5

huww98 added 2 commits June 11, 2025 00:44

Not consider minReadySeconds for PVCs only update

69039de

not integrating RecoverVolumeExpansionFailure

af71200

huww98 force-pushed the sts-update-claim branch from 937c421 to af71200 Compare June 10, 2025 16:44

huww98 force-pushed the sts-update-claim branch 2 times, most recently from 906006f to 0c4d6b5 Compare June 18, 2025 07:46

change PVC/Pod update order

afd1aaf

huww98 force-pushed the sts-update-claim branch from 0c4d6b5 to afd1aaf Compare June 18, 2025 08:07

huww98 added 7 commits June 18, 2025 16:16

describe volumeClaimUpdatePolicy update behaviour

1216743

discuss more on kubectl rollout undo

a708d07

VolumeClaimUpdateStrategy as a struct

d7238a7

target 1.35

d038f90

add a note about VAC being disabled by default

211f8b6

update PVC even if Pod is already deleted/updated

746ff26

Pod may be deleted externally, e.g. evicted.

don't support OnDelete updateStrategy

d4b37da


		#### Story 5: Asymmetric Replicas

		The replicas of our StatefulSet are not identical, so we still want to update

	The replicas of our StatefulSet are not identical, so we still want to update
	The templates for different replicas of a StatefulSet are not identical, so we still want to update

	we can just use the same pipeline that we are already using to updating the Pod.
	we can just use the same pipeline that we are already using to update the Pod.

StatefulSet Support for Updating Volume Claim Template #1

Are you sure you want to change the base?

StatefulSet Support for Updating Volume Claim Template #1

Conversation

huww98 commented May 18, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

huww98 May 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

huww98 May 22, 2024 •

edited

Loading