Skip to content

Commit 4330ee8

Browse files
[PLAT-1854] Document dynamic resource labels for Cloud (#739)
* Add dynamic resource labels * Update platform-cloud/docs/resource-labels/overview.md Co-authored-by: Justine Geffen <[email protected]> Signed-off-by: Llewellyn vd Berg <[email protected]> * Update platform-cloud/docs/resource-labels/overview.md Co-authored-by: Justine Geffen <[email protected]> Signed-off-by: Llewellyn vd Berg <[email protected]> * Update platform-cloud/docs/resource-labels/overview.md Co-authored-by: Justine Geffen <[email protected]> Signed-off-by: Llewellyn vd Berg <[email protected]> * Update platform-cloud/docs/resource-labels/overview.md Co-authored-by: Justine Geffen <[email protected]> Signed-off-by: Llewellyn vd Berg <[email protected]> * Update platform-cloud/docs/resource-labels/overview.md Co-authored-by: Justine Geffen <[email protected]> Signed-off-by: Llewellyn vd Berg <[email protected]> * Update platform-cloud/docs/resource-labels/overview.md Co-authored-by: Justine Geffen <[email protected]> Signed-off-by: Llewellyn vd Berg <[email protected]> * Update platform-cloud/docs/resource-labels/overview.md Co-authored-by: Justine Geffen <[email protected]> Signed-off-by: Llewellyn vd Berg <[email protected]> * Update platform-cloud/docs/resource-labels/overview.md Signed-off-by: Justine Geffen <[email protected]> * Code review suggestions --------- Signed-off-by: Llewellyn vd Berg <[email protected]> Signed-off-by: Justine Geffen <[email protected]> Co-authored-by: Justine Geffen <[email protected]>
1 parent 84d0e9f commit 4330ee8

File tree

1 file changed

+37
-18
lines changed

1 file changed

+37
-18
lines changed

platform-cloud/docs/resource-labels/overview.md

Lines changed: 37 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -5,16 +5,32 @@ date: "24 Apr 2023"
55
tags: [resource labels, labels]
66
---
77

8-
From version 22.3.0, Seqera supports resource labels in compute environments and other elements of your Seqera instance. This offers a flexible tagging system for annotation and tracking of the cloud services consumed by a run.
9-
Resource labels are sent to the service provider for each cloud compute environment in `key=value` format.
8+
Platform supports resource labels in runs, compute environments, pipelines, and actions. This offers a flexible tagging system for annotation and tracking of the cloud services consumed by a run. Resource labels are sent to the cloud service provider in `key=value` format.
109

1110
Resource labels are applied to elements during:
1211

13-
- Compute environment creation with Batch Forge
14-
- Submission
15-
- Execution
12+
- Compute environment creation
13+
- Workflow submission
14+
- Workflow execution
1615

17-
### Create and apply labels
16+
### Dynamic resource labels
17+
18+
Use dynamic resource labels to tag cloud resources with Platform run and Nextflow session identifiers at workflow submission and execution time. When you provide a variable value in the standard `key=value` resource label format, the corresponding unique value for each pipeline run will be propagated to your cloud provider for all the resources spawned by that particular run.
19+
20+
| Value | Description |
21+
|-----------------|---------------------|
22+
| `${workflowId}` | Platform run ID |
23+
| `${sessionId}` | Nextflow session ID |
24+
25+
For example, to apply labels containing the unique Platform run ID to all the resources spawned by a run, add a dynamic label such as `platform-run=${workflowId}` to your compute environment, pipeline, or manually in the pipeline launch form.
26+
27+
Dynamic resource labels applied at the compute environment or pipeline level are prefilled in the pipeline launch form, and they can be applied or overridden during pipeline launch.
28+
29+
:::info
30+
Because dynamic resource labels are used to tag resources with unique values coresponding to a pipeline run, they are only applied during workflow submission and execution time, and not during compute environment creation. See the **Submission time** and **Execution time** resources listed for each cloud provider on this page.
31+
:::
32+
33+
### Create and apply resource labels
1834

1935
Resource labels can be created, applied, and edited by a workspace admin or owner. When applying a label, users can select from existing labels or add new labels on the fly.
2036

@@ -30,11 +46,11 @@ Once the compute environment has been created, its resource labels cannot be edi
3046

3147
If a resource label is applied to a compute environment, all runs in that compute environment will inherit it. Likewise, all cloud resources generated during the workflow execution will be tagged with the same resource label.
3248

33-
#### Resource labels applied to pipelines, actions, and runs
49+
#### Resource labels applied to runs, pipelines, and actions
3450

3551
Admins can override the default resource labels inherited from the compute environment when creating and editing pipelines, actions, and runs on the fly. The custom resource labels associated with each element will propagate to the associated resources in the cloud environment without altering the default resource labels associated with the compute environment.
3652

37-
When an admin adds or edits the resource labels associated with a pipeline, action, or run, the **submission and execution time** resource labels are altered. This does not affect the resource labels for resources spawned at (compute environment) **creation time**.
53+
When an admin adds or edits the resource labels associated with a pipeline, action, or run, the **submission and execution time** resource labels are altered. This does not affect the resource labels for resources spawned at compute environment **creation time**.
3854

3955
For example, the resource label `name=ce1` is set during AWS Batch compute environment creation. If you create the resource label `pipeline=pipeline1` while creating a pipeline which uses the same AWS Batch compute environment, the EC2 instances associated with that compute environment will still contain only the `name=ce1` label, while the Job Definitions associated with the pipeline will inherit the `pipeline=pipeline1` resource label.
4056

@@ -44,25 +60,28 @@ If a maintainer changes the compute environment associated with a pipeline or ru
4460

4561
Search and filter pipelines and runs using one or more resource labels. The resource label search uses a `label:key=value` format.
4662

47-
### Overview of resource labels in a workspace
63+
### Manage workspace resource labels
64+
65+
Select a workspace's **Settings** tab to view all the resource labels used in that workspace. All users can add resource labels, but only admins can edit or delete them, provided they're not already associated with **any** resource. This applies to resource labels associated with compute environments and runs.
4866

49-
Select a workspace's **Settings** tab to view all the resource labels used in that workspace. Resource labels can only be edited or deleted by admins and only if they're not already associated with **any** resource. This applies to resource labels associated with compute environments and runs. When you add or edit a resource label, you can optionally set the **"Use as default in compute environment form"** option. Workspace default resource labels are prefilled in the **Resource labels** field when creating a new compute environment in that workspace.
67+
When you add or edit a resource label, you can optionally set **Use as default in compute environment form**. Workspace default resource labels are prefilled in the **Resource labels** field when you create a new compute environment in that workspace.
5068

5169
The deletion of a resource label from a workspace has no influence on the cloud environment.
5270

5371
### Resource label propagation to cloud environments
5472

5573
:::note
56-
You can't assign multiple resource labels, using the same key, to the same resource — regardless of whether this option is supported by the destination cloud provider.
74+
- Cloud provider credentials must have the appropriate roles or permissions to tag resources in your environment.
75+
- You can't assign multiple resource labels, using the same key, to the same resource, regardless of whether this option is supported by the destination cloud provider.
5776
:::
5877

59-
Resource labels are only available for cloud environments that use a resource tagging system. Seqera supports AWS, Google Batch, Google Life Sciences, Azure, and Kubernetes. HPC compute environments do not support resource labels.
78+
Resource labels are only available for cloud environments that use a resource tagging system. AWS, Azure, Google, and Kubernetes are supported. HPC compute environments do not support resource labels.
6079

61-
Note that the cloud provider credentials you use must have the appropriate roles or permissions to tag resources in your environment.
80+
When a run is executed in a compute environment with associated resource labels:
81+
- Seqera propagates the labels to the set of resources listed for each provider below.
82+
- Nextflow distributes the labels for the resources spawned at runtime.
6283

63-
When a run is executed in a compute environment with associated resource labels, Seqera Platform propagates the labels to a set of resources (listed below), while Nextflow distributes the labels for the resources spawned at runtime.
64-
65-
If the compute environment is created through Forge, the compute environment will propagate the tags to the resources generated by the Forge execution.
84+
If the compute environment is created with Batch Forge, the compute environment will propagate the tags to the resources generated by the Forge execution.
6685

6786
:::caution
6887
Resource label propagation is one-way and not synchronized with the cloud environment. This means that Seqera attaches tags to cloud resources, but isn't aware if those tags are changed or deleted directly in the cloud environment.
@@ -177,7 +196,7 @@ Currently, tagging with resource labels is not available for the files created d
177196

178197
The following resources will be tagged using the labels associated with the compute environment:
179198

180-
**Forge creation time**
199+
**Compute environment creation time**
181200

182201
- Deployment
183202
- PodTemplate
@@ -198,4 +217,4 @@ The following resources will be tagged using the labels associated with the comp
198217
- A maximum of 25 resource labels can be applied to each resource.
199218
- A maximum of 1000 resource labels can be used in each workspace.
200219

201-
See [here](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set) for more information on Kubernetes object labeling.
220+
See [Syntax and character set](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set) for more information on Kubernetes object labeling.

0 commit comments

Comments
 (0)