Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
74fd3bc
Implement ReadStateBytes + WriteStateBytes
radeksimko Aug 13, 2025
bd2039e
[WIP] - Testing ReadStateBytes and WriteStateBytes (#37464)
SarahFrench Aug 26, 2025
28df663
go: bump version to 1.25 (#37436)
dsa0x Aug 18, 2025
29a82e9
PSS: Add alternative, experimental version of `init` command that dow…
SarahFrench Aug 18, 2025
7a6d8ec
Store the FQN of the provider used in PSS in representations of the p…
SarahFrench Jul 10, 2025
e0dbc86
Add test that hits the code path for adding a state store to a new (o…
SarahFrench Jul 11, 2025
ca338f5
Add test for use of `-reconfigure` flag; show that it hits the code p…
SarahFrench Jul 11, 2025
0c88777
Add test that hits the code path for removing use of a state store, m…
SarahFrench Jul 11, 2025
f175c11
Add test that hits the code path for changing a state store's configu…
SarahFrench Jul 11, 2025
99f6963
Update existing test names to be backend-specific
SarahFrench Jul 11, 2025
368eaad
Add tests that hits the code path for migrating between PSS and backends
SarahFrench Jul 11, 2025
4a8e427
Consolidate PSS-related tests at end of the file
SarahFrench Jul 11, 2025
55b7c86
Make init command output report if a backend or state store is being …
SarahFrench Jul 14, 2025
1e2a403
Only process -backend-config flags for use with backend if the user h…
SarahFrench Jul 14, 2025
ce02b0c
Implement initialising an uninitialised dir with PSS
SarahFrench Jul 14, 2025
3dec5b0
Fix call to `ConfigureProvider` by avoiding use of returned config va…
SarahFrench Aug 7, 2025
37f326b
Add TF version to `ConfigureProviderRequest`
SarahFrench Aug 7, 2025
2136e3a
Update test to assert that the newly-initialised backend is reflected…
SarahFrench Aug 7, 2025
19e735a
Update test to test method's behaviour during init and non-init commands
SarahFrench Aug 8, 2025
146019f
Add godoc comment to `GoVersionFromVersion`
SarahFrench Aug 8, 2025
6e58347
command: Prompt user for default workspace creation
radeksimko Aug 4, 2025
00b55a0
Rename argument to backend and state store init methods to help disti…
SarahFrench Aug 8, 2025
d0bcab9
Fix: avoid updating backend state file if an unhandled error has occu…
SarahFrench Aug 8, 2025
16a4029
Fix missing }
SarahFrench Aug 8, 2025
e442714
Add handling of non-default workspace, remove input prompting for now.
SarahFrench Aug 8, 2025
c9a8d49
Rename state manager var, fix bug where PersistState was called on wr…
SarahFrench Aug 8, 2025
0835d26
Make TODO indicating blocker
SarahFrench Aug 8, 2025
374b488
Add createDefaultWorkspace method
SarahFrench Aug 12, 2025
52aaa5c
Add locks to test
SarahFrench Aug 12, 2025
3c88142
Add `-create-default-workspace` flag to init command, for overriding …
SarahFrench Aug 15, 2025
7d07c77
WIP - updating tests, currently blocked
SarahFrench Aug 15, 2025
66395f2
Fix problem from refactoring
SarahFrench Aug 18, 2025
e7aa81d
Update tests with new flag
SarahFrench Aug 18, 2025
352aeeb
Pivot to creating the default workspace without prompts
SarahFrench Aug 18, 2025
44d5dba
Fix: Make sure diags from attempting to create the default workspace …
SarahFrench Aug 26, 2025
eebc38c
Add rough implementation of Get and Put methods on remote gRPC state …
SarahFrench Aug 26, 2025
b84fd8e
Update tests and mock; this is unblocked now Get and Put are implemented
SarahFrench Aug 26, 2025
39c209c
Move init command test cases up to the 'command testing' level. This …
SarahFrench Aug 29, 2025
60cb773
Update Read/WriteStateBytes RPCs to match https://github.com/hashicor…
SarahFrench Aug 29, 2025
1a80026
Run `make protobuf`
SarahFrench Aug 29, 2025
167ea42
Run `make generate`
SarahFrench Aug 29, 2025
7f987b6
Update use of `proto.ReadStateBytes_ResponseChunk` in tests
SarahFrench Aug 29, 2025
f705ad7
Fix how diagnostics are handled alongside EOF error, update ReadState…
SarahFrench Aug 29, 2025
4852374
More fixes - test setup was incorrect
SarahFrench Aug 29, 2025
38d027b
Merge branch 'sarah/update-proto-read-write' into pss/init-of-new-pss…
SarahFrench Aug 29, 2025
972f3b3
Pull code for obtaining provider factory into a separate method
SarahFrench Aug 29, 2025
46aa16e
Make Backend method return diagnostics, so warnings aren't swallowed.
SarahFrench Aug 29, 2025
d334199
Make test assert that warning about skipping workspace creation is in…
SarahFrench Aug 29, 2025
526e66f
Update warning detail to include calls to action
SarahFrench Aug 29, 2025
ffcc2c7
Fix use of hashes when initializing a new workingdir with a state store
SarahFrench Sep 5, 2025
6a01324
Update test to assert backend state file contents
SarahFrench Sep 5, 2025
265d370
Replace test for reconfiguring a state_store when the configuration h…
SarahFrench Sep 5, 2025
8531320
Fix test, add more assertions about output
SarahFrench Sep 5, 2025
d3dc842
Update comments on tests
SarahFrench Sep 8, 2025
f8c3dbb
Fix diagnostic using an error unnecessarily
SarahFrench Sep 15, 2025
d439604
Fix - make sure all errors are handled, and avoid nil error causing a…
SarahFrench Sep 15, 2025
e84e15b
Allow init commands to succeed if using PSS and a reattached provider.
SarahFrench Sep 15, 2025
9b7a292
Fix: Pass state bytes into WriteStateBytesRequest
SarahFrench Sep 16, 2025
1a72b50
Don't try to access chunk when processing an EOF
SarahFrench Sep 17, 2025
cae7619
Add more logging in ReadStateBytes, make sure stream is closed by cor…
SarahFrench Sep 17, 2025
0e4fb67
Make error message more specific
SarahFrench Sep 19, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changes/v1.14/UPGRADE NOTES-20250814-162650.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
kind: UPGRADE NOTES
body: 'The parallelism of Terraform operations within container runtimes may be reduced depending on the CPU bandwidth limit setting.'
time: 2025-08-14T16:26:50.569878+02:00
custom:
Issue: "37436"
5 changes: 5 additions & 0 deletions .changes/v1.14/UPGRADE NOTES-20250814-162752.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
kind: UPGRADE NOTES
body: 'Building Terraform 1.14 requires macOS Monterey or later (due to being built on Go 1.25 which imposes these requirements)'
time: 2025-08-14T16:27:52.659896+02:00
custom:
Issue: "37436"
2 changes: 1 addition & 1 deletion .go-version
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.24.5
1.25
39 changes: 38 additions & 1 deletion docs/plugin-protocol/tfplugin6.proto
Original file line number Diff line number Diff line change
Expand Up @@ -425,6 +425,11 @@ service Provider {
// ConfigureStateStore configures the state store, such as S3 connection in the context of already configured provider
rpc ConfigureStateStore(ConfigureStateStore.Request) returns (ConfigureStateStore.Response);

// ReadStateBytes streams byte chunks of a given state file from a state store
rpc ReadStateBytes(ReadStateBytes.Request) returns (stream ReadStateBytes.Response);
// WriteStateBytes streams byte chunks of a given state file into a state store
rpc WriteStateBytes(stream WriteStateBytes.RequestChunk) returns (WriteStateBytes.Response);

// GetStates returns a list of all states (i.e. CE workspaces) managed by a given state store
rpc GetStates(GetStates.Request) returns (GetStates.Response);
// DeleteState instructs a given state store to delete a specific state (i.e. a CE workspace)
Expand Down Expand Up @@ -918,7 +923,6 @@ message ValidateListResourceConfig {
}
}


message ValidateStateStore {
message Request {
string type_name = 1;
Expand All @@ -939,6 +943,39 @@ message ConfigureStateStore {
}
}

message ReadStateBytes {
message Request {
string type_name = 1;
string state_id = 2;
}
message Response {
bytes bytes = 1;
int64 total_length = 2;
StateRange range = 3;
repeated Diagnostic diagnostics = 4;
}
}

message WriteStateBytes {
message RequestChunk {
// TODO: Can we decouple this outside of the stream?
string type_name = 1;
string state_id = 3;

bytes bytes = 2;
int64 total_length = 4;
StateRange range = 5;
}
message Response {
repeated Diagnostic diagnostics = 1;
}
}

message StateRange {
int64 start = 1;
int64 end = 2;
}

message GetStates {
message Request {
string type_name = 1;
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/hashicorp/terraform

go 1.24.5
go 1.25

godebug winsymlink=0

Expand Down
9 changes: 9 additions & 0 deletions internal/backend/pluggable/pluggable.go
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,15 @@ func (p *Pluggable) ConfigSchema() *configschema.Block {
return val.Body
}

// ProviderSchema returns the schema for the provider implementing the state store.
//
// This isn't part of the backend.Backend interface but is needed in calling code.
// When it's used the backend.Backend will need to be cast to a Pluggable.
func (p *Pluggable) ProviderSchema() *configschema.Block {
schemaResp := p.provider.GetProviderSchema()
return schemaResp.Provider.Body
}

// PrepareConfig validates configuration for the state store in
// the state storage provider. The configuration sent from Terraform core
// will not include any values from environment variables; it is the
Expand Down
2 changes: 1 addition & 1 deletion internal/backend/remote-state/azure/go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/hashicorp/terraform/internal/backend/remote-state/azure

go 1.24.5
go 1.25

require (
github.com/hashicorp/go-azure-helpers v0.72.0
Expand Down
2 changes: 1 addition & 1 deletion internal/backend/remote-state/consul/go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/hashicorp/terraform/internal/backend/remote-state/consul

go 1.24.5
go 1.25

require (
github.com/hashicorp/consul/api v1.32.1
Expand Down
2 changes: 1 addition & 1 deletion internal/backend/remote-state/cos/go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/hashicorp/terraform/internal/backend/remote-state/cos

go 1.24.5
go 1.25

require (
github.com/hashicorp/terraform v0.0.0-00010101000000-000000000000
Expand Down
2 changes: 1 addition & 1 deletion internal/backend/remote-state/gcs/go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/hashicorp/terraform/internal/backend/remote-state/gcs

go 1.24.5
go 1.25

require (
cloud.google.com/go/kms v1.15.5
Expand Down
2 changes: 1 addition & 1 deletion internal/backend/remote-state/kubernetes/go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/hashicorp/terraform/internal/backend/remote-state/kubernetes

go 1.24.5
go 1.25

require (
github.com/hashicorp/terraform v0.0.0-00010101000000-000000000000
Expand Down
2 changes: 1 addition & 1 deletion internal/backend/remote-state/oci/go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/hashicorp/terraform/internal/backend/remote-state/oci

go 1.24.5
go 1.25

require (
github.com/google/go-cmp v0.7.0
Expand Down
2 changes: 1 addition & 1 deletion internal/backend/remote-state/oss/go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/hashicorp/terraform/internal/backend/remote-state/oss

go 1.24.5
go 1.25

require (
github.com/aliyun/alibaba-cloud-sdk-go v1.61.1501
Expand Down
2 changes: 1 addition & 1 deletion internal/backend/remote-state/pg/go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/hashicorp/terraform/internal/backend/remote-state/pg

go 1.24.5
go 1.25

require (
github.com/hashicorp/go-uuid v1.0.3
Expand Down
2 changes: 1 addition & 1 deletion internal/backend/remote-state/s3/go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/hashicorp/terraform/internal/backend/remote-state/s3

go 1.24.5
go 1.25

require (
github.com/aws/aws-sdk-go-v2 v1.36.0
Expand Down
12 changes: 12 additions & 0 deletions internal/builtin/providers/terraform/provider.go
Original file line number Diff line number Diff line change
Expand Up @@ -291,6 +291,18 @@ func (p *Provider) ConfigureStateStore(req providers.ConfigureStateStoreRequest)
return resp
}

func (p *Provider) ReadStateBytes(req providers.ReadStateBytesRequest) providers.ReadStateBytesResponse {
var resp providers.ReadStateBytesResponse
resp.Diagnostics.Append(fmt.Errorf("unsupported state store type %q", req.TypeName))
return resp
}

func (p *Provider) WriteStateBytes(req providers.WriteStateBytesRequest) providers.WriteStateBytesResponse {
var resp providers.WriteStateBytesResponse
resp.Diagnostics.Append(fmt.Errorf("unsupported state store type %q", req.TypeName))
return resp
}

func (p *Provider) GetStates(req providers.GetStatesRequest) providers.GetStatesResponse {
var resp providers.GetStatesResponse
resp.Diagnostics.Append(fmt.Errorf("unsupported state store type %q", req.TypeName))
Expand Down
14 changes: 13 additions & 1 deletion internal/command/arguments/init.go
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,10 @@ type Init struct {
// TODO(SarahFrench/radeksimko): Remove this once the feature is no longer
// experimental
EnablePssExperiment bool

// CreateDefaultWorkspace indicates whether the default workspace should be created by
// Terraform when initializing a state store for the first time.
CreateDefaultWorkspace bool
}

// ParseInit processes CLI arguments, returning an Init value and errors.
Expand Down Expand Up @@ -111,7 +115,7 @@ func ParseInit(args []string) (*Init, tfdiags.Diagnostics) {
cmdFlags.BoolVar(&init.Json, "json", false, "json")
cmdFlags.Var(&init.BackendConfig, "backend-config", "")
cmdFlags.Var(&init.PluginPath, "plugin-dir", "plugin directory")

cmdFlags.BoolVar(&init.CreateDefaultWorkspace, "create-default-workspace", true, "when -input=false, use this flag to block creation of the default workspace")
// Used for enabling experimental code that's invoked before configuration is parsed.
cmdFlags.BoolVar(&init.EnablePssExperiment, "enable-pluggable-state-storage-experiment", false, "Enable the pluggable state storage experiment")

Expand Down Expand Up @@ -139,6 +143,14 @@ func ParseInit(args []string) (*Init, tfdiags.Diagnostics) {
))
}

if init.InputEnabled && !init.CreateDefaultWorkspace {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Warning,
"Invalid init options",
"The flag -create-default-workspace=false is ignored when Terraform is configured to ask users for input. Instead, add -input=false or remove the -create-default-workspace flag",
))
}

init.Args = cmdFlags.Args()

backendFlagSet := FlagIsSet(cmdFlags, "backend")
Expand Down
31 changes: 17 additions & 14 deletions internal/command/arguments/init_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,10 +40,11 @@ func TestParseInit_basicValid(t *testing.T) {
FlagName: "-backend-config",
Items: &flagNameValue,
},
Vars: &Vars{},
InputEnabled: true,
CompactWarnings: false,
TargetFlags: nil,
Vars: &Vars{},
InputEnabled: true,
CompactWarnings: false,
TargetFlags: nil,
CreateDefaultWorkspace: true,
},
},
"setting multiple options": {
Expand Down Expand Up @@ -72,11 +73,12 @@ func TestParseInit_basicValid(t *testing.T) {
FlagName: "-backend-config",
Items: &flagNameValue,
},
Vars: &Vars{},
InputEnabled: true,
Args: []string{},
CompactWarnings: true,
TargetFlags: nil,
Vars: &Vars{},
InputEnabled: true,
Args: []string{},
CompactWarnings: true,
TargetFlags: nil,
CreateDefaultWorkspace: true,
},
},
"with cloud option": {
Expand All @@ -101,11 +103,12 @@ func TestParseInit_basicValid(t *testing.T) {
FlagName: "-backend-config",
Items: &[]FlagNameValue{{Name: "-backend-config", Value: "backend.config"}},
},
Vars: &Vars{},
InputEnabled: false,
Args: []string{},
CompactWarnings: false,
TargetFlags: []string{"foo_bar.baz"},
Vars: &Vars{},
InputEnabled: false,
Args: []string{},
CompactWarnings: false,
TargetFlags: []string{"foo_bar.baz"},
CreateDefaultWorkspace: true,
},
},
}
Expand Down
2 changes: 1 addition & 1 deletion internal/command/cloud_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ func TestCloud_withBackendConfig(t *testing.T) {

// Initialize the backend
ic := &InitCommand{
Meta{
Meta: Meta{
Ui: ui,
View: view,
testingOverrides: metaOverridesForProvider(testProvider()),
Expand Down
Loading
Loading