v0.16.0-beta.0

• GKE: Allow controlling NAT gateway IP addresses and ports per VM #213 - thanks PentoHQ and @Spazzy757
• AKS: Remove deprecated end_date_relative attribute #220
• GKE: Fix preemptible, auto_repair and auto_upgrade node pool attributes #221 - thanks @gullitmiranda
• EKS: Default to separate control plane and node pool subnets #222
• EKS: Add convenience node pool module #224
• GKE: Add convenience node pool module #229
• GKE: Fix deprecated provider attributes #231
• EKS: Support NLB in ingress-dns module #228 - thanks @markszabo
• AKS: Allow configuring whether log analytics is enabled or not #232 - thanks @to266
• AKS: Expose only critical addons taint option for default node pool #238 - thanks @to266
• EKS: Allow setting cluster endpoint access controls #234 - thanks @markszabo
• GKE: Allow setting cluster endpoint access controls #239 - thanks @markszabo
• Update CLI versions, including Terraform to v1.0.11, als fixes az CLI jsmin error #240
• AKS: Add convenience node pool module #235 - thanks @to266
• AKS: Remove in-cluster-module provider configuration #244
• AKS: Expose cluster upgrade opt-in and make channel configurable #241 - thanks @to266
• Update CLI versions, most importantly Terraform to v1.0.11 #242
• AKS: Remove in-cluster azurerm provider configuration (features attribute) #244
• EKS: Fix DNS host recreation due to depends_on #246

Upgrade Notes

Provider versions

Both the azurerm and the google Terraform providers have breaking changes in their respective Kubernetes cluster resource attributes. This Kubestack release updates the modules to adhere to the latest provider version. As such, you should run terraform init --upgrade to update the locked provider versions in your terraform.lock.hcl file.

Once you do, you will also get a new version of the kustomization provider, which switches the default ID format. This switch was necessary, to allow the provider to handle apiVersion updates like required for Ingress without recreating the resources. But the change of the ID format means the ID changes in the Terraform state, which causes an unnecessary destroy-and-recreate plan.

You have two options:

1. You can follow the instructions in the Kustomization provider docs to terraform state mv the resources, after which, the plan will not try to recreate them anymore.
2. If you prefer to stay in the old ID format for now, you can set legacy_id_format = true in the kustomization provider blocks, usually located in the *_providers.tf files.

EKS

Split control plane and node pool subnets

Previously, the default node pool and the control plane used the same subnets. Additionally, those subnets use a /24 CIDR, which means there is only a low number of IP addresses for the control plane and the default node pool to share.

This release by default switches to separate control plane and node pool subnets. The subnet change requires the node pool to be recreated. When recreating the default node pool is not an option at this point in time, you can retain the previous behavior by setting cluster_vpc_legacy_node_subnets = true.

See the following subnet visualization for the default setup: https://www.davidc.net/sites/default/subnets/subnets.html?network=10.18.0.0&mask=16&division=23.ff4011

DNS host recreation

While not strictly required, this change is a fix for a bug that recreates the Route53 DNS hosts unnecessarily. The depends_on on the module propagates to the data sources, which causes them to only be read on apply, and causes the zone_id in the DNS hosts to be known-after-apply and cause a recreate plan.

To avoid this, remove the depends_on line from the eks_zero_dns_zone module in your eks_zero_ingress.tf file.

AKS

In-module provider configuration

Previously, the AKS modules included an in-module provider configuration to set the provider's required features attribute. However, in-module provider configurations are highly discouraged and in this release Kubestack removed this for the azurerm provider in the AKS module.

This means, during upgrade AKS users have to add the following to their aks_zero_providers.tf:

provider "azurerm" {
  features {}
}

Azure log analytics

Making the Azure log analytics configurable for the AKS module required making the azurerm_log_analytics_workspace and azurerm_log_analytics_solution resources conditional using count. This change may show up as a one-time outside of Terraform change when upgrading existing AKS configurations to Kubestack version v0.16.0-beta.0. Log analytics can be enabled using enable_log_analytics = true, the default, or disabled using enable_log_analytics = false.