From f60926ad970bb6e048608926cd27947229eb2c1c Mon Sep 17 00:00:00 2001 From: Llewellyn vd Berg <113503285+llewellyn-sl@users.noreply.github.com> Date: Wed, 23 Jul 2025 15:30:13 -0400 Subject: [PATCH 01/10] Add dynamic resource labels --- .../docs/resource-labels/overview.md | 48 ++++++++++++------- 1 file changed, 30 insertions(+), 18 deletions(-) diff --git a/platform-cloud/docs/resource-labels/overview.md b/platform-cloud/docs/resource-labels/overview.md index 035ca6445..494728489 100644 --- a/platform-cloud/docs/resource-labels/overview.md +++ b/platform-cloud/docs/resource-labels/overview.md @@ -5,16 +5,27 @@ date: "24 Apr 2023" tags: [resource labels, labels] --- -From version 22.3.0, Seqera supports resource labels in compute environments and other elements of your Seqera instance. This offers a flexible tagging system for annotation and tracking of the cloud services consumed by a run. -Resource labels are sent to the service provider for each cloud compute environment in `key=value` format. +Platform supports resource labels in runs, compute environments, pipelines, and actions. This offers a flexible tagging system for annotation and tracking of the cloud services consumed by a run. Resource labels are sent to the cloud service provider in `key=value` format. Resource labels are applied to elements during: -- Compute environment creation with Batch Forge -- Submission -- Execution +- Compute environment creation +- Workflow submission +- Workflow execution -### Create and apply labels +### Dynamic resource labels + +Use dynamic resource labels to tag cloud resources with Platform run and Nextflow session identifiers at workflow submission and execution time. By providing a variable value (either `${workflowId}` or `${sessionId}`) in the standard `key=value` resource label format, the unique run or session ID for each pipeline run will be propagated to your cloud provider for all the resources spawned by that particular run. + +For example, for user Alex to apply labels containing the unique Platform run ID to all the resources spawned by a run, they could apply a dynamic label such as `alex=run-${workflowId}` to their compute environment, pipeline, or manually in the pipeline launch form. + +Dynamic resource labels applied at the compute environment or pipeline level are prefilled in the pipeline launch form, and they can be applied or overridden during pipeline launch. + +:::info +Since dynamic resource labels are used to tag the resources spawned with the Platform run ID or Nextflow session ID, they are only applied during workflow submission and execution time and not at compute environment creation. Therefore, only the **Submission time** and **Execution time** resources listed for each cloud provider on this page can be tagged with a dynamic resource label. +::: + +### Create and apply resource labels Resource labels can be created, applied, and edited by a workspace admin or owner. When applying a label, users can select from existing labels or add new labels on the fly. @@ -30,11 +41,11 @@ Once the compute environment has been created, its resource labels cannot be edi If a resource label is applied to a compute environment, all runs in that compute environment will inherit it. Likewise, all cloud resources generated during the workflow execution will be tagged with the same resource label. -#### Resource labels applied to pipelines, actions, and runs +#### Resource labels applied to runs, pipelines, and actions Admins can override the default resource labels inherited from the compute environment when creating and editing pipelines, actions, and runs on the fly. The custom resource labels associated with each element will propagate to the associated resources in the cloud environment without altering the default resource labels associated with the compute environment. -When an admin adds or edits the resource labels associated with a pipeline, action, or run, the **submission and execution time** resource labels are altered. This does not affect the resource labels for resources spawned at (compute environment) **creation time**. +When an admin adds or edits the resource labels associated with a pipeline, action, or run, the **submission and execution time** resource labels are altered. This does not affect the resource labels for resources spawned at compute environment **creation time**. For example, the resource label `name=ce1` is set during AWS Batch compute environment creation. If you create the resource label `pipeline=pipeline1` while creating a pipeline which uses the same AWS Batch compute environment, the EC2 instances associated with that compute environment will still contain only the `name=ce1` label, while the Job Definitions associated with the pipeline will inherit the `pipeline=pipeline1` resource label. @@ -44,25 +55,26 @@ If a maintainer changes the compute environment associated with a pipeline or ru Search and filter pipelines and runs using one or more resource labels. The resource label search uses a `label:key=value` format. -### Overview of resource labels in a workspace +### Manage workspace resource labels -Select a workspace's **Settings** tab to view all the resource labels used in that workspace. Resource labels can only be edited or deleted by admins and only if they're not already associated with **any** resource. This applies to resource labels associated with compute environments and runs. When you add or edit a resource label, you can optionally set the **"Use as default in compute environment form"** option. Workspace default resource labels are prefilled in the **Resource labels** field when creating a new compute environment in that workspace. +Select a workspace's **Settings** tab to view all the resource labels used in that workspace. Resource labels can only be edited or deleted by admins and only if they're not already associated with **any** resource. This applies to resource labels associated with compute environments and runs. When you add or edit a resource label, you can optionally set the **Use as default in compute environment form** option. Workspace default resource labels are prefilled in the **Resource labels** field when creating a new compute environment in that workspace. The deletion of a resource label from a workspace has no influence on the cloud environment. ### Resource label propagation to cloud environments :::note -You can't assign multiple resource labels, using the same key, to the same resource — regardless of whether this option is supported by the destination cloud provider. +- Cloud provider credentials must have the appropriate roles or permissions to tag resources in your environment. +- You can't assign multiple resource labels, using the same key, to the same resource — regardless of whether this option is supported by the destination cloud provider. ::: -Resource labels are only available for cloud environments that use a resource tagging system. Seqera supports AWS, Google Batch, Google Life Sciences, Azure, and Kubernetes. HPC compute environments do not support resource labels. +Resource labels are only available for cloud environments that use a resource tagging system. AWS, Azure, Google Cloud, and Kubernetes are supported. HPC compute environments do not support resource labels. -Note that the cloud provider credentials you use must have the appropriate roles or permissions to tag resources in your environment. +When a run is executed in a compute environment with associated resource labels: +- Seqera propagates the labels to the set of resources listed for each provider below +- Nextflow distributes the labels for the resources spawned at runtime. -When a run is executed in a compute environment with associated resource labels, Seqera Platform propagates the labels to a set of resources (listed below), while Nextflow distributes the labels for the resources spawned at runtime. - -If the compute environment is created through Forge, the compute environment will propagate the tags to the resources generated by the Forge execution. +If the compute environment is created with Batch Forge, the compute environment will propagate the tags to the resources generated by the Forge execution. :::caution Resource label propagation is one-way and not synchronized with the cloud environment. This means that Seqera attaches tags to cloud resources, but isn't aware if those tags are changed or deleted directly in the cloud environment. @@ -177,7 +189,7 @@ Currently, tagging with resource labels is not available for the files created d The following resources will be tagged using the labels associated with the compute environment: -**Forge creation time** +**Compute environment creation time** - Deployment - PodTemplate @@ -198,4 +210,4 @@ The following resources will be tagged using the labels associated with the comp - A maximum of 25 resource labels can be applied to each resource. - A maximum of 1000 resource labels can be used in each workspace. -See [here](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set) for more information on Kubernetes object labeling. +See [Syntax and character set](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set) for more information on Kubernetes object labeling. From 3e8c5c54c2a084837663e21f2449ce3607c18376 Mon Sep 17 00:00:00 2001 From: Llewellyn vd Berg <113503285+llewellyn-sl@users.noreply.github.com> Date: Thu, 24 Jul 2025 17:08:40 +0200 Subject: [PATCH 02/10] Update platform-cloud/docs/resource-labels/overview.md Co-authored-by: Justine Geffen Signed-off-by: Llewellyn vd Berg <113503285+llewellyn-sl@users.noreply.github.com> --- platform-cloud/docs/resource-labels/overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/platform-cloud/docs/resource-labels/overview.md b/platform-cloud/docs/resource-labels/overview.md index 494728489..f74e70030 100644 --- a/platform-cloud/docs/resource-labels/overview.md +++ b/platform-cloud/docs/resource-labels/overview.md @@ -15,7 +15,7 @@ Resource labels are applied to elements during: ### Dynamic resource labels -Use dynamic resource labels to tag cloud resources with Platform run and Nextflow session identifiers at workflow submission and execution time. By providing a variable value (either `${workflowId}` or `${sessionId}`) in the standard `key=value` resource label format, the unique run or session ID for each pipeline run will be propagated to your cloud provider for all the resources spawned by that particular run. +Use dynamic resource labels to tag cloud resources with Platform run and Nextflow session identifiers at workflow submission and execution time. When you provide a variable value (either `${workflowId}` or `${sessionId}`) in the standard `key=value` resource label format, the unique run or session ID for each pipeline run will be propagated to your cloud provider for all the resources spawned by that particular run. For example, for user Alex to apply labels containing the unique Platform run ID to all the resources spawned by a run, they could apply a dynamic label such as `alex=run-${workflowId}` to their compute environment, pipeline, or manually in the pipeline launch form. From e31d80ea13476c14f2328105cdddb3f7b5b9ed48 Mon Sep 17 00:00:00 2001 From: Llewellyn vd Berg <113503285+llewellyn-sl@users.noreply.github.com> Date: Thu, 24 Jul 2025 17:09:05 +0200 Subject: [PATCH 03/10] Update platform-cloud/docs/resource-labels/overview.md Co-authored-by: Justine Geffen Signed-off-by: Llewellyn vd Berg <113503285+llewellyn-sl@users.noreply.github.com> --- platform-cloud/docs/resource-labels/overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/platform-cloud/docs/resource-labels/overview.md b/platform-cloud/docs/resource-labels/overview.md index f74e70030..031ee4b57 100644 --- a/platform-cloud/docs/resource-labels/overview.md +++ b/platform-cloud/docs/resource-labels/overview.md @@ -17,7 +17,7 @@ Resource labels are applied to elements during: Use dynamic resource labels to tag cloud resources with Platform run and Nextflow session identifiers at workflow submission and execution time. When you provide a variable value (either `${workflowId}` or `${sessionId}`) in the standard `key=value` resource label format, the unique run or session ID for each pipeline run will be propagated to your cloud provider for all the resources spawned by that particular run. -For example, for user Alex to apply labels containing the unique Platform run ID to all the resources spawned by a run, they could apply a dynamic label such as `alex=run-${workflowId}` to their compute environment, pipeline, or manually in the pipeline launch form. +For example, to apply labels containing the unique Platform run ID to all the resources spawned by a run, add a dynamic label such as `alex=run-${workflowId}` to your compute environment, pipeline, or manually in the pipeline launch form. Dynamic resource labels applied at the compute environment or pipeline level are prefilled in the pipeline launch form, and they can be applied or overridden during pipeline launch. From c3d65f4437a0c5469f0da3dc4a19d32529b6fb39 Mon Sep 17 00:00:00 2001 From: Llewellyn vd Berg <113503285+llewellyn-sl@users.noreply.github.com> Date: Thu, 24 Jul 2025 17:09:12 +0200 Subject: [PATCH 04/10] Update platform-cloud/docs/resource-labels/overview.md Co-authored-by: Justine Geffen Signed-off-by: Llewellyn vd Berg <113503285+llewellyn-sl@users.noreply.github.com> --- platform-cloud/docs/resource-labels/overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/platform-cloud/docs/resource-labels/overview.md b/platform-cloud/docs/resource-labels/overview.md index 031ee4b57..de38fe1f4 100644 --- a/platform-cloud/docs/resource-labels/overview.md +++ b/platform-cloud/docs/resource-labels/overview.md @@ -19,7 +19,7 @@ Use dynamic resource labels to tag cloud resources with Platform run and Nextflo For example, to apply labels containing the unique Platform run ID to all the resources spawned by a run, add a dynamic label such as `alex=run-${workflowId}` to your compute environment, pipeline, or manually in the pipeline launch form. -Dynamic resource labels applied at the compute environment or pipeline level are prefilled in the pipeline launch form, and they can be applied or overridden during pipeline launch. +Dynamic resource labels applied at the compute environment or pipeline level are prefilled in the pipeline launch form, and they can be applied or overridden during pipeline launch. :::info Since dynamic resource labels are used to tag the resources spawned with the Platform run ID or Nextflow session ID, they are only applied during workflow submission and execution time and not at compute environment creation. Therefore, only the **Submission time** and **Execution time** resources listed for each cloud provider on this page can be tagged with a dynamic resource label. From a05d782199051654be71fe9872a97851a67f27ba Mon Sep 17 00:00:00 2001 From: Llewellyn vd Berg <113503285+llewellyn-sl@users.noreply.github.com> Date: Thu, 24 Jul 2025 17:09:34 +0200 Subject: [PATCH 05/10] Update platform-cloud/docs/resource-labels/overview.md Co-authored-by: Justine Geffen Signed-off-by: Llewellyn vd Berg <113503285+llewellyn-sl@users.noreply.github.com> --- platform-cloud/docs/resource-labels/overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/platform-cloud/docs/resource-labels/overview.md b/platform-cloud/docs/resource-labels/overview.md index de38fe1f4..9a7087d1f 100644 --- a/platform-cloud/docs/resource-labels/overview.md +++ b/platform-cloud/docs/resource-labels/overview.md @@ -22,7 +22,7 @@ For example, to apply labels containing the unique Platform run ID to all the re Dynamic resource labels applied at the compute environment or pipeline level are prefilled in the pipeline launch form, and they can be applied or overridden during pipeline launch. :::info -Since dynamic resource labels are used to tag the resources spawned with the Platform run ID or Nextflow session ID, they are only applied during workflow submission and execution time and not at compute environment creation. Therefore, only the **Submission time** and **Execution time** resources listed for each cloud provider on this page can be tagged with a dynamic resource label. +Because dynamic resource labels are used to tag the resources spawned with the Platform run ID or Nextflow session ID, they are only applied during workflow submission and execution time, and not at compute environment creation. As a result, only the **Submission time** and **Execution time** resources listed for each cloud provider on this page can be tagged with a dynamic resource label. ::: ### Create and apply resource labels From 02916e642bc2898f611eccf2ef38fd2f1e8fbaf0 Mon Sep 17 00:00:00 2001 From: Llewellyn vd Berg <113503285+llewellyn-sl@users.noreply.github.com> Date: Thu, 24 Jul 2025 17:10:44 +0200 Subject: [PATCH 06/10] Update platform-cloud/docs/resource-labels/overview.md Co-authored-by: Justine Geffen Signed-off-by: Llewellyn vd Berg <113503285+llewellyn-sl@users.noreply.github.com> --- platform-cloud/docs/resource-labels/overview.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/platform-cloud/docs/resource-labels/overview.md b/platform-cloud/docs/resource-labels/overview.md index 9a7087d1f..fd0f9fb51 100644 --- a/platform-cloud/docs/resource-labels/overview.md +++ b/platform-cloud/docs/resource-labels/overview.md @@ -57,7 +57,9 @@ Search and filter pipelines and runs using one or more resource labels. The reso ### Manage workspace resource labels -Select a workspace's **Settings** tab to view all the resource labels used in that workspace. Resource labels can only be edited or deleted by admins and only if they're not already associated with **any** resource. This applies to resource labels associated with compute environments and runs. When you add or edit a resource label, you can optionally set the **Use as default in compute environment form** option. Workspace default resource labels are prefilled in the **Resource labels** field when creating a new compute environment in that workspace. +Select a workspace's **Settings** tab to view all the resource labels used in that workspace. All users can add resource labels, but only admins can edit or delete them, provided they're not already associated with **any** resource. This applies to resource labels associated with compute environments and runs. + +When you add or edit a resource label, you can optionally set **Use as default in compute environment form**. Workspace default resource labels are prefilled in the **Resource labels** field when you create a new compute environment in that workspace. The deletion of a resource label from a workspace has no influence on the cloud environment. From f84fe2b1fa68b1832538ffce14679c6fc5220e97 Mon Sep 17 00:00:00 2001 From: Llewellyn vd Berg <113503285+llewellyn-sl@users.noreply.github.com> Date: Thu, 24 Jul 2025 17:11:02 +0200 Subject: [PATCH 07/10] Update platform-cloud/docs/resource-labels/overview.md Co-authored-by: Justine Geffen Signed-off-by: Llewellyn vd Berg <113503285+llewellyn-sl@users.noreply.github.com> --- platform-cloud/docs/resource-labels/overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/platform-cloud/docs/resource-labels/overview.md b/platform-cloud/docs/resource-labels/overview.md index fd0f9fb51..945d7921d 100644 --- a/platform-cloud/docs/resource-labels/overview.md +++ b/platform-cloud/docs/resource-labels/overview.md @@ -67,7 +67,7 @@ The deletion of a resource label from a workspace has no influence on the cloud :::note - Cloud provider credentials must have the appropriate roles or permissions to tag resources in your environment. -- You can't assign multiple resource labels, using the same key, to the same resource — regardless of whether this option is supported by the destination cloud provider. +- You can't assign multiple resource labels, using the same key, to the same resource, regardless of whether this option is supported by the destination cloud provider. ::: Resource labels are only available for cloud environments that use a resource tagging system. AWS, Azure, Google Cloud, and Kubernetes are supported. HPC compute environments do not support resource labels. From afcce3e5f980b9d7249818d8090f1ddf643bd177 Mon Sep 17 00:00:00 2001 From: Llewellyn vd Berg <113503285+llewellyn-sl@users.noreply.github.com> Date: Thu, 24 Jul 2025 17:11:16 +0200 Subject: [PATCH 08/10] Update platform-cloud/docs/resource-labels/overview.md Co-authored-by: Justine Geffen Signed-off-by: Llewellyn vd Berg <113503285+llewellyn-sl@users.noreply.github.com> --- platform-cloud/docs/resource-labels/overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/platform-cloud/docs/resource-labels/overview.md b/platform-cloud/docs/resource-labels/overview.md index 945d7921d..4f32e57a9 100644 --- a/platform-cloud/docs/resource-labels/overview.md +++ b/platform-cloud/docs/resource-labels/overview.md @@ -73,7 +73,7 @@ The deletion of a resource label from a workspace has no influence on the cloud Resource labels are only available for cloud environments that use a resource tagging system. AWS, Azure, Google Cloud, and Kubernetes are supported. HPC compute environments do not support resource labels. When a run is executed in a compute environment with associated resource labels: -- Seqera propagates the labels to the set of resources listed for each provider below +- Seqera propagates the labels to the set of resources listed for each provider below. - Nextflow distributes the labels for the resources spawned at runtime. If the compute environment is created with Batch Forge, the compute environment will propagate the tags to the resources generated by the Forge execution. From b00b6ac9e51ccb8151b7d784be3a23cda8ef2b74 Mon Sep 17 00:00:00 2001 From: Justine Geffen Date: Thu, 24 Jul 2025 19:37:32 +0200 Subject: [PATCH 09/10] Update platform-cloud/docs/resource-labels/overview.md Signed-off-by: Justine Geffen --- platform-cloud/docs/resource-labels/overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/platform-cloud/docs/resource-labels/overview.md b/platform-cloud/docs/resource-labels/overview.md index 4f32e57a9..82d83d2a4 100644 --- a/platform-cloud/docs/resource-labels/overview.md +++ b/platform-cloud/docs/resource-labels/overview.md @@ -17,7 +17,7 @@ Resource labels are applied to elements during: Use dynamic resource labels to tag cloud resources with Platform run and Nextflow session identifiers at workflow submission and execution time. When you provide a variable value (either `${workflowId}` or `${sessionId}`) in the standard `key=value` resource label format, the unique run or session ID for each pipeline run will be propagated to your cloud provider for all the resources spawned by that particular run. -For example, to apply labels containing the unique Platform run ID to all the resources spawned by a run, add a dynamic label such as `alex=run-${workflowId}` to your compute environment, pipeline, or manually in the pipeline launch form. +For example, to apply labels containing the unique Platform run ID to all the resources spawned by a run, add a dynamic label such as `name=run-${workflowId}` to your compute environment, pipeline, or manually in the pipeline launch form. Dynamic resource labels applied at the compute environment or pipeline level are prefilled in the pipeline launch form, and they can be applied or overridden during pipeline launch. From 40334e3e2a57b2fc0c7572d3bb35404990af095f Mon Sep 17 00:00:00 2001 From: Llewellyn vd Berg <113503285+llewellyn-sl@users.noreply.github.com> Date: Tue, 5 Aug 2025 14:40:13 -0400 Subject: [PATCH 10/10] Code review suggestions --- platform-cloud/docs/resource-labels/overview.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/platform-cloud/docs/resource-labels/overview.md b/platform-cloud/docs/resource-labels/overview.md index 82d83d2a4..dfc08d046 100644 --- a/platform-cloud/docs/resource-labels/overview.md +++ b/platform-cloud/docs/resource-labels/overview.md @@ -15,14 +15,19 @@ Resource labels are applied to elements during: ### Dynamic resource labels -Use dynamic resource labels to tag cloud resources with Platform run and Nextflow session identifiers at workflow submission and execution time. When you provide a variable value (either `${workflowId}` or `${sessionId}`) in the standard `key=value` resource label format, the unique run or session ID for each pipeline run will be propagated to your cloud provider for all the resources spawned by that particular run. +Use dynamic resource labels to tag cloud resources with Platform run and Nextflow session identifiers at workflow submission and execution time. When you provide a variable value in the standard `key=value` resource label format, the corresponding unique value for each pipeline run will be propagated to your cloud provider for all the resources spawned by that particular run. -For example, to apply labels containing the unique Platform run ID to all the resources spawned by a run, add a dynamic label such as `name=run-${workflowId}` to your compute environment, pipeline, or manually in the pipeline launch form. +| Value | Description | +|-----------------|---------------------| +| `${workflowId}` | Platform run ID | +| `${sessionId}` | Nextflow session ID | + +For example, to apply labels containing the unique Platform run ID to all the resources spawned by a run, add a dynamic label such as `platform-run=${workflowId}` to your compute environment, pipeline, or manually in the pipeline launch form. Dynamic resource labels applied at the compute environment or pipeline level are prefilled in the pipeline launch form, and they can be applied or overridden during pipeline launch. :::info -Because dynamic resource labels are used to tag the resources spawned with the Platform run ID or Nextflow session ID, they are only applied during workflow submission and execution time, and not at compute environment creation. As a result, only the **Submission time** and **Execution time** resources listed for each cloud provider on this page can be tagged with a dynamic resource label. +Because dynamic resource labels are used to tag resources with unique values coresponding to a pipeline run, they are only applied during workflow submission and execution time, and not during compute environment creation. See the **Submission time** and **Execution time** resources listed for each cloud provider on this page. ::: ### Create and apply resource labels @@ -70,7 +75,7 @@ The deletion of a resource label from a workspace has no influence on the cloud - You can't assign multiple resource labels, using the same key, to the same resource, regardless of whether this option is supported by the destination cloud provider. ::: -Resource labels are only available for cloud environments that use a resource tagging system. AWS, Azure, Google Cloud, and Kubernetes are supported. HPC compute environments do not support resource labels. +Resource labels are only available for cloud environments that use a resource tagging system. AWS, Azure, Google, and Kubernetes are supported. HPC compute environments do not support resource labels. When a run is executed in a compute environment with associated resource labels: - Seqera propagates the labels to the set of resources listed for each provider below.