Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions docs/deploying_clearml/enterprise_deploy/appgw.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
---
title: AI Application Gateway
title: Application Gateway
---

:::important Enterprise Feature
The AI Application Gateway is available under the ClearML Enterprise plan.
The Application Gateway is available under the ClearML Enterprise plan.
:::

Services running through a cluster orchestrator such as Kubernetes or cloud hyperscaler require meticulous configuration
to make them available as these environments do not expose their networks to external users.

The ClearML AI Application Gateway facilitates setting up secure, authenticated access to jobs running on your compute
The ClearML Application Gateway facilitates setting up secure, authenticated access to jobs running on your compute
nodes from external networks.

Using the AI Application Gateway, services are allocated externally accessible, SSL secure network routes which provide
access in adherence to ClearML RBAC privileges. The AI Application Gateway supports HTTP/S as well as raw TCP routing.
Using the Application Gateway, services are allocated externally accessible, SSL secure network routes which provide
access in adherence to ClearML RBAC privileges. The Application Gateway supports HTTP/S as well as raw TCP routing.

The following ClearML UI applications make use of the AI Application Gateway to provide authenticated HTTPS access to
The following ClearML UI applications make use of the Application Gateway to provide authenticated HTTPS access to
their instances:

* GPUaaS
Expand All @@ -31,7 +31,7 @@ their instances:
* [Llama.cpp Model Deployment](../../webapp/applications/apps_llama_deployment.md)
* [LLM UI](../../webapp/applications/apps_llm_ui.md)

The AI Application Gateway requires an additional component to the ClearML Server deployment: the **ClearML App Gateway Router**.
The Application Gateway requires an additional component to the ClearML Server deployment: the **ClearML App Gateway Router**.
If your ClearML Deployment does not have the App Gateway Router properly installed, these application instances may not be accessible.

#### Installation
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,14 @@ title: Docker-Compose Deployment
---

:::important Enterprise Feature
The AI Application Gateway is available under the ClearML Enterprise plan.
The Application Gateway is available under the ClearML Enterprise plan.
:::

The AI Application Gateway enables external HTTP(S) or direct TCP access to ClearML tasks and applications running on
The Application Gateway enables external HTTP(S) or direct TCP access to ClearML tasks and applications running on
nodes. The gateway is configured with an endpoint or external address, making these services accessible from the user's
machine, outside the workload's network.

This guide describes how to install and run the ClearML AI Application Gateway using docker-compose for environments
This guide describes how to install and run the ClearML Application Gateway using docker-compose for environments
where you manage both the ClearML Server and the workload nodes.


Expand Down Expand Up @@ -97,7 +97,7 @@ TCP_PORT_END=
```

**Configuration Options:**
* `PROXY_TAG`: AI Application Gateway proxy tag. The Docker image tag for the proxy component, which needs to be
* `PROXY_TAG`: Application Gateway proxy tag. The Docker image tag for the proxy component, which needs to be
specified during installation. This tag is provided by ClearML to ensure compatibility with the recommended version.
* `ROUTER_TAG`: App Gateway Router tag. The Docker image tag for the router component. It defines the specific version
to be installed and is provided by ClearML as part of the setup process.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@ title: Docker-Compose - Hosted Server
---

:::important Enterprise Feature
The AI Application Gateway is available under the ClearML Enterprise plan.
The Application Gateway is available under the ClearML Enterprise plan.
:::

The AI Application Gateway enables external access to ClearML tasks, and applications running on workload nodes that
The Application Gateway enables external access to ClearML tasks, and applications running on workload nodes that
require HTTP or TCP access. The gateway is configured with an endpoint or external address, making these services
accessible from the user's machine, outside the workload nodes’ network.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Kubernetes Deployment
---

:::important Enterprise Feature
The AI Application Gateway is available under the ClearML Enterprise plan.
The Application Gateway is available under the ClearML Enterprise plan.
:::

This guide details the installation of the ClearML App Gateway.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@
title: Deployment with Self-Signed Certificates
---

This guide covers how to configure the [AI Application Gateway](../appgw.md) and [ClearML Agent](../../../clearml_agent/clearml_agent_deployment_k8s.md#agent-with-an-enterprise-server)
This guide covers how to configure the [Application Gateway](../appgw.md) and [ClearML Agent](../../../clearml_agent/clearml_agent_deployment_k8s.md#agent-with-an-enterprise-server)
to use self-signed or custom SSL certificates.

## Certificate Configuration

To configure certificates, update the applicable overrides file:
* For AI Application Gateway: `clearml-app-gateway-values.override.yaml` file
* For Application Gateway: `clearml-app-gateway-values.override.yaml` file
* For ClearML Agent: `clearml-agent-values.override.yaml` file

```yaml
Expand Down Expand Up @@ -121,7 +121,7 @@ Their names are usually prefixed with the Helm release name, so adjust according
### Apply Changes

To apply the changes, run the update command:
* For AI Application Gateway:
* For Application Gateway:

```bash
helm upgrade -i <RELEASE_NAME> -n <WORKLOAD_NAMESPACE> clearml-enterprise/clearml-enterprise-app-gateway --version <CHART_VERSION> -f clearml-app-gateway-values.override.yaml
Expand Down
8 changes: 4 additions & 4 deletions docs/deploying_clearml/enterprise_deploy/k8s.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ To deploy a ClearML Server, ensure the following components and configurations a
- Ingress Controller: An Ingress controller (e.g., `nginx-ingress`) is required. If exposing services externally, configure
LoadBalancer-capable solution (e.g. `MetalLB`).
- Server and workers that communicate on HTTP/S (ports 80 and 443). Additionally, the TCP session feature requires a
range of ports for TCP traffic based on your configuration (see [AI App Gateway installation](appgw_install_k8s.md)).
range of ports for TCP traffic based on your configuration (see [Application Gateway installation](appgw_install_k8s.md)).
- DNS Configuration: A domain with subdomain support is required, ideally with trusted TLS certificates. All entries must
be resolvable by the Ingress controller. Example subdomains:
- Server:
Expand Down Expand Up @@ -182,12 +182,12 @@ The ClearML Enterprise Agent enables scheduling and execution of distributed wor

See the [ClearML Agent installation guide](../../clearml_agent/clearml_agent_deployment_k8s.md#agent-with-an-enterprise-server).

### AI Application Gateway
### Application Gateway

The AI App Gateway enables secure, authenticated access to ClearML application endpoints such as model serving or IDE workloads
The Application Gateway enables secure, authenticated access to ClearML application endpoints such as model serving or IDE workloads
based on ClearML user permissions. It routes HTTPS traffic from users to running pods on the cluster.

See the [AI Application Gateway installation guide](appgw_install_k8s.md).
See the [Application Gateway installation guide](appgw_install_k8s.md).

## Advanced Options
### GPU Operator
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,7 @@ described [above](#create-a-tenant).
In multi-tenant setups, you can separate the tenant workers in different namespaces:

* Create a Kubernetes Namespace for each tenant
* Deploy a dedicated ClearML Agent and AI Application Gateway in each Namespace.
* Deploy a dedicated ClearML Agent and Application Gateway in each Namespace.
* Configure a tenant Agent and Gateway with credentials created on the ClearML Server by a user of the same tenant.

Additional network separation can be achieved via Kubernetes Network Policies.
Expand Down
6 changes: 3 additions & 3 deletions docs/webapp/applications/apps_embed_model_deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,12 @@ cases. The Embedding Model Deployment application serves your model on a machine
running, it serves your embedding model through a secure, publicly accessible network endpoint. The app monitors
endpoint activity and shuts down if the model remains inactive for a specified maximum idle time.

:::info AI Application Gateway
:::info Application Gateway
The Embedding Model Deployment app makes use of the App Gateway Router which implements a secure, authenticated
network endpoint for the model.

If the ClearML AI application Gateway is not available, the model endpoint might not be accessible.
For more information, see [AI Application Gateway](../../deploying_clearml/enterprise_deploy/appgw.md).
If the ClearML Application Gateway is not available, the model endpoint might not be accessible.
For more information, see [Application Gateway](../../deploying_clearml/enterprise_deploy/appgw.md).
:::

After starting an Embedding Model Deployment instance, you can view the following information in its dashboard:
Expand Down
6 changes: 3 additions & 3 deletions docs/webapp/applications/apps_gradio.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,12 @@ The Gradio launcher monitors the Gradio app activity and shuts down if it is ina

<a id="traffic_router"/>

:::important AI Application Gateway
:::important Application Gateway
The Gradio Launcher relies on the ClearML App Gateway Router which implements user authentication, and redirects requests
to the IP/port served by the Gradio app.

If the ClearML AI application Gateway is not available, the Gradio app might not be accessible.
For more information, see [AI Application Gateway](../../deploying_clearml/enterprise_deploy/appgw.md).
If the ClearML Application Gateway is not available, the Gradio app might not be accessible.
For more information, see [Application Gateway](../../deploying_clearml/enterprise_deploy/appgw.md).
:::

Once you start a Gradio launcher instance, you can view the following information in its dashboard:
Expand Down
6 changes: 3 additions & 3 deletions docs/webapp/applications/apps_llama_deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@ The llama.cpp Model Deployment application serves your model on a machine of you
running, it serves your model through a secure, publicly accessible network endpoint. The app monitors endpoint activity
and shuts down if the model remains inactive for a specified maximum idle time.

:::important AI Application Gateway
:::important Application Gateway
The llama.cpp Model Deployment app makes use of the App Gateway Router which implements a secure, authenticated
network endpoint for the model.

If the ClearML AI application Gateway is not available, the model endpoint might not be accessible.
For more information, see [AI Application Gateway](../../deploying_clearml/enterprise_deploy/appgw.md).
If the ClearML Application Gateway is not available, the model endpoint might not be accessible.
For more information, see [Application Gateway](../../deploying_clearml/enterprise_deploy/appgw.md).
:::

After starting a llama.cpp Model Deployment instance, you can view the following information in its dashboard:
Expand Down
6 changes: 3 additions & 3 deletions docs/webapp/applications/apps_llm_ui.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ max tokens, and stop sequences. For detailed usage and customization options, se
[Open WebUI documentation ](https://docs.openwebui.com/).


:::info AI Application Gateway
:::info Application Gateway
The LLM UI app makes use of the App Gateway Router which implements a secure, authenticated network endpoint for the
model. If the ClearML AI application Gateway is not available, the model endpoint might not be accessible. For more
information, see [AI Application Gateway](../../deploying_clearml/enterprise_deploy/appgw.md).
model. If the ClearML Application Gateway is not available, the model endpoint might not be accessible. For more
information, see [Application Gateway](../../deploying_clearml/enterprise_deploy/appgw.md).
:::

After launching an LLM UI instance, its dashboard displays the following:
Expand Down
6 changes: 3 additions & 3 deletions docs/webapp/applications/apps_model_deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,12 @@ usage. The vLLM Model Deployment application serves your model on a machine of y
it serves your model through a secure, publicly accessible network endpoint. The app monitors endpoint activity and
shuts down if the model remains inactive for a specified maximum idle time.

:::info AI Application Gateway
:::info Application Gateway
The vLLM Model Deployment app makes use of the App Gateway Router which implements a secure, authenticated
network endpoint for the model.

If the ClearML AI application Gateway is not available, the model endpoint might not be accessible.
For more information, see [AI Application Gateway](../../deploying_clearml/enterprise_deploy/appgw.md).
If the ClearML Application Gateway is not available, the model endpoint might not be accessible.
For more information, see [Application Gateway](../../deploying_clearml/enterprise_deploy/appgw.md).
:::

Once you start a vLLM Model Deployment instance, you can view the following information in its dashboard:
Expand Down
6 changes: 3 additions & 3 deletions docs/webapp/applications/apps_streamlit.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,12 @@ time.

<a id="traffic_router"/>

:::important AI Application Gateway
:::important Application Gateway
The Streamlit Launcher relies on the ClearML App Gateway Router which implements user authentication, and redirects requests
to the IP/port served by the Streamlit app.

If the ClearML AI application Gateway is not available, the Streamlit app might not be accessible.
For more information, see [AI Application Gateway](../../deploying_clearml/enterprise_deploy/appgw.md).
If the ClearML Application Gateway is not available, the Streamlit app might not be accessible.
For more information, see [Application Gateway](../../deploying_clearml/enterprise_deploy/appgw.md).
:::

Once you start a Streamlit launcher instance, you can view the following information in its dashboard:
Expand Down
4 changes: 2 additions & 2 deletions docs/webapp/settings/webapp_settings_app_gw.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: Application Gateway
---

The ClearML [AI Application Gateway](../../deploying_clearml/enterprise_deploy/appgw.md) facilitates setting up secure,
The ClearML [Application Gateway](../../deploying_clearml/enterprise_deploy/appgw.md) facilitates setting up secure,
authenticated access to jobs running on your compute nodes from external networks (see application gateway installation
for [Kubernetes](../../deploying_clearml/enterprise_deploy/appgw_install_k8s.md), Docker-Compose for [Self-Hosted Deployment](../../deploying_clearml/enterprise_deploy/appgw_install_compose.md)
or Docker-Compose for [Hosted Deployment](../../deploying_clearml/enterprise_deploy/appgw_install_compose_hosted.md)).
Expand All @@ -13,7 +13,7 @@ or Docker-Compose for [Hosted Deployment](../../deploying_clearml/enterprise_dep

## Routers

The **Routers** table lets you monitor all active application gateway routers, as well as verify gateway functionality. The table shows each routers:
The **Routers** table lets you monitor all active app gateway routers, as well as verify gateway functionality. The table shows each router's:
* Name
* Externally accessible URL
* Test Status: The result of the most recent connectivity test
Expand Down
8 changes: 4 additions & 4 deletions docs/webapp/settings/webapp_settings_profile.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,17 +97,17 @@ these credentials cannot be recovered.

**To revoke ClearML credentials:** hover over the desired credentials, and click <img src="/docs/latest/icons/ico-trash.svg" alt="Trash can" className="icon size-md" /> .

### AI Application Gateway Tokens
### Application Gateway Tokens

:::important Enterprise Feature
The AI Application Gateway is available under the ClearML Enterprise plan.
The Application Gateway is available under the ClearML Enterprise plan.
:::

The [AI Application Gateway](../../deploying_clearml/enterprise_deploy/appgw.md) enables external access to ClearML
The [Application Gateway](../../deploying_clearml/enterprise_deploy/appgw.md) enables external access to ClearML
tasks and applications. The gateway is configured with an
endpoint or external address (ingress), accessible from outside ClearML.

Generate tokens providing API access to the AI Application Gateway endpoints:
Generate tokens providing API access to the Application Gateway endpoints:

1. Click **Generate a Token**
1. Under `Expiration`, enter the number of days the token should remain valid
Expand Down