Skip to content

Commit 365ade9

Browse files
authored
Lifecycle management - data, decommission, track (#930)
2 parents 7cdea2b + 7289d07 commit 365ade9

File tree

5 files changed

+265
-2
lines changed

5 files changed

+265
-2
lines changed

CODEOWNERS

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,3 +34,6 @@
3434
# Sentinel documentation ownership
3535
/content/sentinel/ @hashicorp/team-docs-packer-and-terraform @hashicorp/tf-compliance
3636

37+
# Well-architected framework
38+
39+
/content/well-architected-framework/ @hashicorp/well-architected-education-approvers

content/well-architected-framework/data/docs-nav-data.json

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -415,6 +415,19 @@
415415
"title": "Overview",
416416
"path": "optimize-systems"
417417
},
418+
{
419+
"title": "Lifecycle management",
420+
"routes": [
421+
{
422+
"title": "Implement data retention policies",
423+
"path": "optimize-systems/lifecycle-management/data-management"
424+
},
425+
{
426+
"title": "Decommission resources",
427+
"path": "optimize-systems/lifecycle-management/decommission-infrastructure"
428+
}
429+
]
430+
},
418431
{
419432
"title": "Monitor system health",
420433
"routes": [

content/well-architected-framework/docs/docs/define-and-automate-processes/automate/cicd.mdx

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -31,5 +31,6 @@ In this section of Automate your workflows, you learned how to implement CI/CD p
3131

3232
Visit the following documents to learn more about the automation workflow:
3333

34-
- [Automate testing](/well-architected-framework/define-and-automate-processes/automate/testing) - Implement automated testing in your CI/CD pipeline
35-
- [Automate deployments](/well-architected-framework/define-and-automate-processes/automate/deployments) - Deploy applications through your CI/CD pipeline
34+
- [Automate testing](/well-architected-framework/define-and-automate-processes/automate/testing) in your CI/CD pipelines
35+
- [Automate application deployments](/well-architected-framework/define-and-automate-processes/automate/deployments through your CI/CD pipeline
36+
- Learn how to orchestrate [Terraform runs](/terraform/tutorials/automation/automate-terraform) to ensure consistency between runs.
Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
---
2+
page_title: Implement data management policies
3+
description: Implement data management policies to reduce storage costs, ensure compliance, and manage data lifecycles with infrastructure as code.
4+
---
5+
6+
# Implement data management policies
7+
8+
You can use data management policies to manage the lifecycle of your organization's data. When you store data either in the cloud or on-premises, it is important to define and automate the policies around managing that data. Defining management with infrastructure as code tools, such as Terraform, ensures you consistently apply these policies across all environments and resources.
9+
10+
## Why you should use lifecycle policies
11+
12+
Most major cloud providers offer lifecycle management features for their storage services. These features allow you to define rules that automatically transition data between different storage classes based on age or access patterns, and delete data that has reached the end of its retention period.
13+
14+
When you implement data management policies, you gain the following benefits:
15+
- Reduce storage costs by automatically deleting data that is no longer needed.
16+
- Reduce storage costs by storing data in the most cost-effective storage class based on access patterns and retention requirements.
17+
- Ensure compliance with legal and regulatory requirements for data retention.
18+
- Minimize security risks by removing sensitive data after a defined period of time.
19+
20+
## Automate policy management with infrastructure as code
21+
22+
You can use Terraform to define and manage lifecycle policies and implement those policies across your organization. You can create Terraform modules to create data management policies for different data types and compliance requirements. These modules can automatically apply appropriate lifecycle rules, storage class transitions, and deletion policies to new or existing storage resources.
23+
24+
The following Terraform configuration defines a [data lifecycle policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_lifecycle_configuration#specifying-a-filter-based-on-object-size) to move AWS S3 data to Glacier Instant Retrieval after 365 days:
25+
26+
```hcl
27+
resource "aws_s3_bucket_lifecycle_configuration" "example" {
28+
bucket = aws_s3_bucket.bucket.id
29+
30+
rule {
31+
id = "Allow small object transitions"
32+
33+
filter {
34+
object_size_greater_than = 1
35+
}
36+
37+
status = "Enabled"
38+
39+
transition {
40+
days = 365
41+
storage_class = "GLACIER_IR"
42+
}
43+
}
44+
}
45+
```
46+
47+
Terraform can also tag resources with appropriate retention metadata. These tags can include creation dates, data classifications, and retention periods.
48+
49+
For example, you can use the [`tag` block](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_lifecycle_configuration#specifying-a-filter-based-on-an-object-tag) with AWS S3 to automatically apply tags to all resources created by Terraform. The S3 lifecycle rule specifies a filter based on a tag key and value. The rule then applies only to a subset of objects with the specific tag.
50+
51+
```hcl
52+
resource "aws_s3_bucket_lifecycle_configuration" "example" {
53+
bucket = aws_s3_bucket.bucket.id
54+
55+
rule {
56+
id = "rule-1"
57+
58+
filter {
59+
tag {
60+
key = "Name"
61+
value = "Staging"
62+
}
63+
}
64+
65+
transition {
66+
days = 30
67+
storage_class = "GLACIER"
68+
}
69+
70+
status = "Enabled"
71+
}
72+
}
73+
```
74+
75+
Other cloud providers, such as [Google Cloud Platform](https://registry.terraform.io/providers/hashicorp/google/5.0.0/docs/resources/storage_bucket.html#example-usage---life-cycle-settings-for-storage-bucket-objects) and [Microsoft Azure](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/storage_management_policy), offer similar lifecycle management features for their storage services. You can use Terraform to manage lifecycle policies across multiple cloud providers, ensuring consistent data management practices regardless of where your data resides.
76+
77+
HashiCorp resources:
78+
79+
- Search the [Terraform Registry](https://registry.terraform.io/browse/providers) for the [cloud](https://registry.terraform.io/browse/providers?category=public-cloud) or [database](https://registry.terraform.io/browse/providers?category=database) provider you use.
80+
81+
- Learn best practices for writing Terraform with the Terraform [style guide](/terraform/language/style).
82+
83+
External resources:
84+
85+
- Cloud storage: [AWS](https://aws.amazon.com/products/storage/), [GCP](https://cloud.google.com/products/storage), and [Azure](https://azure.microsoft.com/en-us/products/category/storage)
86+
- [Learn how to set the lifecycle configuration for a Google Cloud Bucket](https://cloud.google.com/storage/docs/samples/storage-create-lifecycle-setting-tf) with Terraform.
87+
- AWS [Enforce data retention policies](https://docs.aws.amazon.com/wellarchitected/latest/framework/cost_decomissioning_resources_data_retention.html)
88+
89+
## Next steps
90+
91+
In this section of Lifecycle management, you learned about implementing data management policies, including why you should use lifecycle policies and how to automate policy management with infrastructure as code. Implement data management policies is part of the [Optimize systems](/well-architected-framework/optimize-systems) pillar.
92+
93+
To learn more about infrastructure and resource management, refer to the following resources:
94+
- [Automate infrastructure provisioning](/well-architected-framework/define-and-automate-processes/process-automation/process-automation-workflow)
Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
---
2+
page_title: Decommission resources
3+
description: Learn how to decommission infrastructure components while maintaining system integrity and avoiding disruptions through proper planning and automation.
4+
---
5+
6+
# Decommission resources
7+
8+
Resource decommissioning is the process of safely removing or deleting infrastructure components, applications, or services that are no longer needed or have reached end-of-life. You should remove unused or obsolete resources such as servers, databases, images, IAM, and other infrastructure components.
9+
10+
When you decommission unused resources, you gain the following benefits:
11+
- Reduce costs by removing charges associated with unused resources.
12+
- Minimize security risks by removing outdated or vulnerable resources that bad actors can exploit.
13+
- Reduce configuration drift by only running necessary resources.
14+
- Improve audit and compliance by maintaining a smaller infrastructure footprint.
15+
16+
To successfully decommission resources, you need to create a well-defined plan that includes dependency analysis, stakeholder communication, and a gradual removal process. Depending on how your infrastructure implementation is done, either manually or automatically, you may need to adjust your decommissioning approach.
17+
18+
## Find resources to decommission
19+
20+
Before you begin decommissioning resources, you need to identify which resources exist in your environment and determine which ones are candidates for removal. This discovery phase helps you avoid accidentally removing resources that are still in use and ensures you target the right components for decommissioning.
21+
22+
Start by creating an inventory of your infrastructure. Most cloud providers offer resource tagging and billing reports that help identify unused or underutilized resources. Pay particular attention to active resources created for temporary purposes, like testing or proof-of-concepts.
23+
24+
Terraform tracks all infrastructure it manages with state files. You can use the `terraform state list` to see all managed resources and `terraform show` to examine their current configurations. This list of resources will help you identify which resources are still in use and which ones you can decommission.
25+
26+
If you're using HCP Terraform, you can use the [workspace explorer](/terraform/cloud-docs/workspaces/explorer) feature to gain visibility into the resThat ources your organization manages with Terraform. The explorer provides a visual representation of your infrastructure, making it easier to identify resources that you no longer need.
27+
28+
## Create a dependency plan
29+
30+
Your plan should analyze which services, applications, or other resources rely on the components you plan to remove. Your plan will lower the risk of unexpected outages by identifying and addressing dependencies before decommissioning.
31+
32+
If you are using infrastructure as code tools like Terraform, you can use a dependency graph to understand resource relationships. This graph can help you visualize connections between resources and identify potential impacts of removing specific components.
33+
34+
The following command creates a dependency graph of your Terraform resources:
35+
36+
```shell-session
37+
$ terraform graph -type=plan | dot -Tpng > graph.png
38+
```
39+
40+
<Note>
41+
42+
You need to install Graphviz on your system to use the `terraform graph` command and generate visualizations. For more information on installing Graphviz, refer to the [Graphviz installation guide](https://graphviz.org/download/).
43+
44+
</Note>
45+
46+
HashiCorp resources:
47+
48+
- [Terraform graph command](/terraform/cli/commands/graph)
49+
50+
## Create a communication plan
51+
52+
Your plan should outline how you will inform stakeholders about the decommissioning process, including timelines and potential impacts. Effective communication prevents surprises and ensures all affected teams can prepare for the changes.
53+
54+
Start by identifying all stakeholders who might be affected by the decommissioning, including development teams, operations staff, end users, and business owners. Create a notification timeline that provides adequate warning. Your communications should explain what resources you are removing, when the decommissioning will occur, and what actions stakeholders need to take.
55+
56+
## Create backups
57+
58+
Before decommissioning, confirm that you have backups of any critical data or configurations associated with the resources you are removing. Backups provide a safety net in case you need to roll back changes.
59+
60+
You may want to back up the following resources:
61+
- Servers in the form of machine images
62+
- Database snapshots
63+
- Configuration files
64+
- Metadata
65+
66+
Since Terraform uses infrastructure as code to manage resources, you can redeploy resources that you have previously decommissioned by reapplying your Terraform configuration. This capability allows you to recover resources quickly if needed.
67+
68+
For example, if you backed up a server, you can also redeploy it by updating the AMI in your Terraform with the backed-up AMI ID. In the following example, you can change the `ami` attribute to the ID of your backed-up AMI:
69+
70+
```hcl
71+
resource "aws_instance" "example" {
72+
ami = "ami-0c55b159cbfafe1f0"
73+
instance_type = "t2.micro"
74+
}
75+
```
76+
77+
You can also use Terraform to create [AWS EBS snapshots](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ebs_snapshot) before decommissioning instances. The following example creates an EBS snapshot of the root volume of an EC2 instance:
78+
79+
```hcl
80+
resource "aws_ebs_volume" "example" {
81+
availability_zone = "us-west-2a"
82+
size = 40
83+
84+
tags = {
85+
Name = "HelloWorld"
86+
}
87+
}
88+
89+
resource "aws_ebs_snapshot" "example_snapshot" {
90+
volume_id = aws_ebs_volume.example.id
91+
92+
tags = {
93+
Name = "HelloWorld_snap"
94+
}
95+
}
96+
```
97+
98+
## Gradually remove resources
99+
100+
Implement a phased approach to removing resources instead of doing it all at once. Start by redirecting traffic away from the resource, and monitor user traffic to ensure you don't negatively impact users.
101+
102+
You can use `terraform plan` to preview the changes that will occur when you remove resources from your configuration. This command helps you understand the impact of your changes before applying them.
103+
104+
You can also set safeguards so you only decommission resources when you are ready. You can use Terraform's `lifecycle` block with `prevent_destroy = true` to prevent accidental deletion of critical resources. The [lifecycle](/terraform/language/meta-arguments#lifecycle) setting ensures that you won't destroy resources unless you explicitly remove the `prevent_destroy` attribute.
105+
106+
```hcl
107+
resource "aws_instance" "example" {
108+
ami = "ami-0c55b159cbfafe1f0"
109+
instance_type = "t2.micro"
110+
}
111+
112+
lifecycle {
113+
prevent_destroy = true
114+
}
115+
```
116+
117+
Consul can help you gradually remove resources by directing traffic away from services you are decommissioning. You can use Consul's service discovery and health checking features to monitor the status of services and ensure that dependent services are not affected during the decommissioning process.
118+
119+
If you are using orchestration tools like Nomad or Kubernetes, you can use their built-in capabilities to drain workloads before decommissioning nodes gracefully. Nomad provides node drain functionality through the `nomad node drain` command, which prevents new scheduling new allocations on a node while safely migrating existing jobs to other available nodes. The Kubernetes `kubectl drain` command safely removes pods from nodes while respecting Pod Disruption Budgets, which ensure that a minimum number of application replicas remain available throughout the process.
120+
121+
HashiCorp resources:
122+
123+
- Review the [Zero-downtime deployments](/well-architected-framework/define-and-automate-processes/deploy/zero-downtime-deployments) documentation for strategies on how to redirect traffic and disable functions gradually.
124+
- Learn how to [manage resource lifecycles with Terraform](/terraform/tutorials/state/resource-lifecycle).
125+
- [Get up and running with Nomad](/nomad/tutorials/get-started) by learning about scheduling, setting up a cluster, and deploying an example job.
126+
- Learn the [fundamentals of Consul](/consul/tutorials).
127+
128+
## Verify health of infrastructure and applications
129+
130+
After the decommissioning process, verify that the remaining infrastructure and applications are functioning correctly. Monitor system performance and user feedback to ensure that there are no negative impacts.
131+
132+
You should do the following steps after you decomission the resources:
133+
134+
- Validate APIs are functioning.
135+
- Check application performance.
136+
- Monitor system logs for errors.
137+
138+
HashiCorp resources:
139+
140+
- [Learn to setup monitoring agents](/well-architected-framework/define-and-automate-processes/monitor/setup-monitoring-agents) and [dashboards and alerts](/well-architected-framework/define-and-automate-processes/monitor/dashboards-alerts).
141+
142+
External resources:
143+
144+
- AWS [Implement a decommissioning process](https://docs.aws.amazon.com/wellarchitected/latest/framework/cost_decomissioning_resources_implement_process.html)
145+
146+
## Next steps
147+
148+
In this section of Lifecycle management, you learned about decommissioning resources, including why you should plan decommissioning and how to safely execute the process. Decommission resources is part of the [Optimize systems](/well-architected-framework/optimize-systems) pillar.
149+
150+
To learn more about infrastructure and resource management, refer to the following resource:
151+
152+
- [Data management](/well-architected-framework/optimize-systems/lifecycle-management/data-management)

0 commit comments

Comments
 (0)