Skip to content

Commit e9d16b0

Browse files
k8s-ci-robotk8s-ci-robot
authored
Merge pull request #2319 from sedefsavas/multitenancy-docs
[docs] Add multitenancy documentation
2 parents 4111016 + 5746097 commit e9d16b0

File tree

3 files changed

+328
-56
lines changed

3 files changed

+328
-56
lines changed

‎docs/book/src/SUMMARY_PREFIX.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
- [Using clusterawsadm to fulfill prerequisites](./topics/using-clusterawsadm-to-fulfill-prerequisites.md)
88
- [Accessing EC2 instances](./topics/accessing-ec2-instances.md)
99
- [Machine Pools](./topics/machinepools.md)
10+
- [Multi-tenancy](./topics/multitenancy.md)
1011
- [EKS Support](./topics/eks/index.md)
1112
- [Prerequisites](./topics/eks/prerequisites.md)
1213
- [Enabling EKS Support](./topics/eks/enabling.md)

‎docs/book/src/topics/multitenancy.md‎

Lines changed: 271 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,271 @@
1+
# Multi-tenancy
2+
3+
Starting from v0.6.5, single controller multi-tenancy is supported that allows using a different AWS Identity for each workload cluster.
4+
For details, see the [multi-tenancy proposal](https://github.com/kubernetes-sigs/cluster-api-provider-aws/blob/master/docs/proposal/20200506-single-controller-multitenancy.md).
5+
6+
7+
For multi-tenancy support, a reference field (`identityRef`) is added to `AWSCluster`, which describes the identity to be used when reconciling the cluster.
8+
9+
```yaml
10+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
11+
kind: AWSCluster
12+
metadata:
13+
name: "test"
14+
namespace: "test"
15+
spec:
16+
region: "eu-west-1"
17+
identityRef:
18+
kind: <IdentityType>
19+
name: <IdentityName>
20+
```
21+
22+
Identity resources are used to describe IAM identities that will be used during reconciliation.
23+
There are three identity types: AWSClusterControllerIdentity, AWSClusterStaticIdentity, and AWSClusterRoleIdentity.
24+
Once an IAM identity is created in AWS, the corresponding values should be used to create a identity resource.
25+
26+
## AWSClusterControllerIdentity
27+
28+
Before multi-tenancy support, all AWSClusters were being reconciled using the credentials that are used by Cluster API Provider AWS Controllers.
29+
`AWSClusterControllerIdentity` is used to restrict the usage of controller credentials only to AWSClusters that are in `allowedNamespaces`.
30+
Since CAPA controllers use a single set of credentials, `AWSClusterControllerIdentity` is a singleton, and can only be created with `name: default`.
31+
32+
For backward compatibility, `AutoControllerIdentityCreator` experimental feature is added, which is responsible to create the `AWSClusterControllerIdentity` singleton if it does not exist.
33+
- **Feature status:** Experimental
34+
- **Feature gate:** AutoControllerIdentityCreator=true
35+
`AutoControllerIdentityCreator` creates `AWSClusterControllerIdentity` singleton with empty `allowedNamespaces` (allowedNamespaces: {}) to grant access to the `AWSClusterControllerIdentity` from all namespaces.
36+
37+
Example:
38+
```yaml
39+
---
40+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
41+
kind: AWSCluster
42+
metadata:
43+
name: "test"
44+
namespace: "test"
45+
spec:
46+
region: "eu-west-1"
47+
identityRef:
48+
kind: AWSClusterControllerIdentity
49+
name: default
50+
---
51+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
52+
kind: AWSClusterControllerIdentity
53+
metadata:
54+
name: "default"
55+
spec:
56+
allowedNamespaces:{} # matches all namespaces
57+
```
58+
`AWSClusterControllerIdentity` is immutable to avoid any unwanted overrides to the allowed namespaces, especially during upgrading clusters.
59+
60+
## AWSClusterIdentityIdentity
61+
`AWSClusterIdentityIdentity` represents static AWS credentials, which are stored in a `Secret`.
62+
63+
Example: Below, an `AWSClusterIdentityIdentity` is created that allows access to the `AWSClusters` that are in "test" namespace.
64+
The identity credentials that will be used by "test" AWSCluster are stored in "test-account-creds" secret.
65+
66+
67+
```yaml
68+
---
69+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
70+
kind: AWSCluster
71+
metadata:
72+
name: "test"
73+
namespace: "test"
74+
spec:
75+
region: "eu-west-1"
76+
identityRef:
77+
kind: AWSClusterIdentityIdentity
78+
name: test-account
79+
---
80+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
81+
kind: AWSClusterIdentityIdentity
82+
metadata:
83+
name: "test-account"
84+
spec:
85+
secretRef:
86+
name: test-account-creds
87+
namespace: capa-system
88+
allowedNamespaces:
89+
selector:
90+
matchLabels:
91+
ns: "testlabel"
92+
---
93+
apiVersion: v1
94+
kind: Namespace
95+
metadata:
96+
labels:
97+
cluster.x-k8s.io/ns: "testlabel"
98+
name: "test"
99+
---
100+
apiVersion: v1
101+
kind: Secret
102+
metadata:
103+
name: "test-account-creds"
104+
namespace: capa-system
105+
stringData:
106+
accessKeyID: AKIAIOSFODNN7EXAMPLE
107+
secretAccessKey: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
108+
```
109+
110+
## AWSClusterRoleIdentity
111+
`AWSClusterRoleIdentity` allows CAPA to assume a role either in the same or another AWS account, using the STS::AssumeRole API.
112+
The assumed role could be used by the AWSClusters that is in the `allowedNamespaces`.
113+
114+
Example:
115+
Below, an `AWSClusterRoleIdentity` instance, which will be used by AWSCluster "test", is created.
116+
This role will be assumed by the source identity at runtime. Source identity can be of any identity type.
117+
Role is assumed in the beginning once and after, whenever the assumed role's credentials are expired.
118+
119+
```yaml
120+
---
121+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
122+
kind: AWSCluster
123+
metadata:
124+
name: "test"
125+
namespace: "test"
126+
spec:
127+
region: "eu-west-1"
128+
identityRef:
129+
kind: AWSClusterRoleIdentity
130+
name: test-account-role
131+
---
132+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
133+
kind: AWSClusterRoleIdentity
134+
metadata:
135+
name: "test-account-role"
136+
spec:
137+
allowedNamespaces:
138+
list: # allows only "test" namespace to use this identity
139+
"test"
140+
roleARN: "arn:aws:iam::123456789:role/CAPARole"
141+
sourceIdentityRef:
142+
kind: AWSClusterIdentityIdentity
143+
name: test-account-creds
144+
```
145+
146+
Nested role assumption is also supported.
147+
Example: Below, "multi-tenancy-nested-role" will be assumed by "multi-tenancy-role", which will be assumed by the "default" `AWSClusterControllerIdentity`
148+
149+
```yaml
150+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
151+
kind: AWSClusterRoleIdentity
152+
metadata:
153+
name: multi-tenancy-role
154+
spec:
155+
allowedNamespaces:
156+
list: []
157+
durationSeconds: 900 # default and min value is 900 seconds
158+
roleARN: arn:aws:iam::11122233344:role/multi-tenancy-role
159+
sessionName: multi-tenancy-role-session
160+
sourceidentityRef:
161+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
162+
kind: AWSClusterControllerIdentity
163+
name: default
164+
---
165+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
166+
kind: AWSClusterRoleIdentity
167+
metadata:
168+
name: multi-tenancy-nested-role
169+
spec:
170+
allowedNamespaces:
171+
list: []
172+
roleARN: arn:aws:iam::11122233355:role/multi-tenancy-nested-role
173+
sessionName: multi-tenancy-nested-role-session
174+
sourceidentityRef:
175+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
176+
kind: AWSClusterRoleIdentity
177+
name: multi-tenancy-role
178+
```
179+
180+
## Secure Access to Identitys
181+
`allowedNamespaces` field is used to grant access to the namespaces to use Identitys.
182+
Only AWSClusters that are created in one of the Identity's allowed namespaces can use that Identity.
183+
`allowedNamespaces` are defined by providing either a list of namespaces or label selector to select namespaces.
184+
185+
### Examples
186+
187+
An empty `allowedNamespaces` indicates that the Identity can be used by all namespaces.
188+
189+
```yaml
190+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
191+
kind: AWSClusterControllerIdentity
192+
spec:
193+
allowedNamespaces:{} # matches all namespaces
194+
```
195+
196+
Having a nil `list` and a nil `selector` is the same with having an empty `allowedNamespaces` (Identity can be used by all namespaces).
197+
198+
```yaml
199+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
200+
kind: AWSClusterControllerIdentity
201+
spec:
202+
allowedNamespaces:
203+
list: nil
204+
selector: nil
205+
```
206+
207+
A nil `allowedNamespaces` indicates that the Identity cannot be used from any namespace.
208+
209+
```yaml
210+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
211+
kind: AWSClusterControllerIdentity
212+
spec:
213+
allowedNamespaces: # this is same with not providing the field at all or allowedNamespaces: null
214+
```
215+
216+
The union of namespaces that are matched by `selector` and the namespaces that are in the `list` is granted access to the identity.
217+
The namespaces that are not in the list and not matching the selector will not have access.
218+
219+
Nil or empty `list` matches no namespaces. Nil or empty `selector` matches no namespaces.
220+
If `list` is nil and `selector` is empty OR `list` is empty and `selector` is nil, Identity cannot be used from any namespace.
221+
Because in this case, `allowedNamespaces` is not empty or nil, and neither `list` nor `selector` allows any namespaces, so the union is empty.
222+
223+
```yaml
224+
# Matches no namespaces
225+
allowedNamespaces:
226+
list: []
227+
```
228+
```yaml
229+
# Matches no namespaces
230+
allowedNamespaces:
231+
selector: {}
232+
```
233+
```yaml
234+
# Matches no namespaces
235+
allowedNamespaces:
236+
list: null
237+
selector: {}
238+
```
239+
```yaml
240+
# Matches no namespaces
241+
allowedNamespaces:
242+
list: []
243+
selector: {}
244+
```
245+
246+
**Important** The default behaviour of an empty label selector is to match all objects, however here we do not follow that behavior to avoid unintended access to the identitys.
247+
This is consistent with core cluster API selectors, e.g., Machine and ClusterResourceSet selectors. The result of matchLabels and matchExpressions are ANDed.
248+
249+
250+
In Kubernetes selectors, `matchLabels` and `matchExpressions` are ANDed.
251+
In the example below, list is empty/nil, so does not allow any namespaces and selector matches with only `default` namespace.
252+
Since `list` and `selector` results are ORed, `default` namespace can use this identity.
253+
254+
```yaml
255+
kind: namespace
256+
metadata:
257+
name: default
258+
labels:
259+
environment: dev
260+
---
261+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
262+
kind: AWSClusterControllerIdentity
263+
spec:
264+
allowedNamespaces:
265+
list: null # or []
266+
selector:
267+
matchLabels:
268+
namespace: default
269+
matchExpressions:
270+
- {key: environment, operator: In, values: [dev]}
271+
```

0 commit comments

Comments
 (0)