Skip to content

Commit e33de3b

Browse files
dmr: consolidate Compose content (#22994)
Consolidate Compose content/remove duplicated information. --------- Co-authored-by: Dorin-Andrei Geman <[email protected]>
1 parent 90b08ad commit e33de3b

File tree

4 files changed

+52
-124
lines changed

4 files changed

+52
-124
lines changed

content/manuals/ai/compose/model-runner.md

Lines changed: 0 additions & 116 deletions
This file was deleted.

content/manuals/ai/compose/models-and-compose.md

Lines changed: 50 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,9 @@ title: Define AI Models in Docker Compose applications
33
linkTitle: Use AI models in Compose
44
description: Learn how to define and use AI models in Docker Compose applications using the models top-level element
55
keywords: compose, docker compose, models, ai, machine learning, cloud providers, specification
6+
alias:
7+
- /compose/how-tos/model-runner/
8+
- /ai/compose/model-runner/
69
weight: 10
710
params:
811
sidebar:
@@ -18,11 +21,12 @@ Compose lets you define AI models as core components of your application, so you
1821
## Prerequisites
1922

2023
- Docker Compose v2.38 or later
21-
- A platform that supports Compose models such as Docker Model Runner or compatible cloud providers
24+
- A platform that supports Compose models such as Docker Model Runner (DMR) or compatible cloud providers.
25+
If you are using DMR, see the [requirements](/manuals/ai/model-runner/_index.md#requirements).
2226

2327
## What are Compose models?
2428

25-
Compose `models` are a standardized way to define AI model dependencies in your application. By using the []`models` top-level element](/reference/compose-file/models.md) in your Compose file, you can:
29+
Compose `models` are a standardized way to define AI model dependencies in your application. By using the [`models` top-level element](/reference/compose-file/models.md) in your Compose file, you can:
2630

2731
- Declare which AI models your application needs
2832
- Specify model configurations and requirements
@@ -66,8 +70,15 @@ models:
6670
Common configuration options include:
6771
- `model` (required): The OCI artifact identifier for the model. This is what Compose pulls and runs via the model runner.
6872
- `context_size`: Defines the maximum token context size for the model.
73+
74+
> [!NOTE]
75+
> Each model has its own maximum context size. When increasing the context length,
76+
> consider your hardware constraints. In general, try to keep context size
77+
> as small as feasible for your specific needs.
78+
6979
- `runtime_flags`: A list of raw command-line flags passed to the inference engine when the model is started.
70-
- Platform-specific options may also be available via extensions attributes `x-*`
80+
For example, if you use llama.cpp, you can pass any of [the available parameters](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md).
81+
- Platform-specific options may also be available via extension attributes `x-*`
7182

7283
## Service model binding
7384

@@ -131,25 +142,58 @@ One of the key benefits of using Compose models is portability across different
131142

132143
### Docker Model Runner
133144

134-
When Docker Model Runner is enabled:
145+
When [Docker Model Runner is enabled](/manuals/ai/model-runner/_index.md):
135146

136147
```yaml
137148
services:
138149
chat-app:
139150
image: my-chat-app
140151
models:
141-
- llm
152+
llm:
153+
endpoint_var: AI_MODEL_URL
154+
model_var: AI_MODEL_NAME
142155
143156
models:
144157
llm:
145158
model: ai/smollm2
159+
context_size: 4096
160+
runtime_flags:
161+
- "--no-prefill-assistant"
146162
```
147163

148164
Docker Model Runner will:
149165
- Pull and run the specified model locally
150166
- Provide endpoint URLs for accessing the model
151167
- Inject environment variables into the service
152168

169+
#### Alternative configuration with provider services
170+
171+
> [!TIP]
172+
>
173+
> This approach is deprecated. Use the [`models` top-level element](#basic-model-definition) instead.
174+
175+
You can also use the `provider` service type, which allows you to declare platform capabilities required by your application.
176+
For AI models, you can use the `model` type to declare model dependencies.
177+
178+
To define a model provider:
179+
180+
```yaml
181+
services:
182+
chat:
183+
image: my-chat-app
184+
depends_on:
185+
- ai_runner
186+
187+
ai_runner:
188+
provider:
189+
type: model
190+
options:
191+
model: ai/smollm2
192+
context-size: 1024
193+
runtime-flags: "--no-prefill-assistant"
194+
```
195+
196+
153197
### Cloud providers
154198

155199
The same Compose file can run on cloud providers that support Compose models:
@@ -181,4 +225,4 @@ Cloud providers might:
181225
- [`models` top-level element](/reference/compose-file/models.md)
182226
- [`models` attribute](/reference/compose-file/services.md#models)
183227
- [Docker Model Runner documentation](/manuals/ai/model-runner.md)
184-
- [Compose Model Runner documentation](/manuals/ai/compose/model-runner.md)
228+
- [Compose Model Runner documentation](/manuals/ai/compose/models-and-compose.md)

content/manuals/ai/model-runner/_index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,7 @@ Models are pulled from Docker Hub the first time they're used and stored locally
8484
> Using Testcontainers or Docker Compose?
8585
> [Testcontainers for Java](https://java.testcontainers.org/modules/docker_model_runner/)
8686
> and [Go](https://golang.testcontainers.org/modules/dockermodelrunner/), and
87-
> [Docker Compose](/manuals/ai/compose/model-runner.md) now support Docker Model Runner.
87+
> [Docker Compose](/manuals/ai/compose/models-and-compose.md) now support Docker Model Runner.
8888
8989
## Enable Docker Model Runner
9090

content/manuals/desktop/release-notes.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -240,7 +240,7 @@ For more frequently asked questions, see the [FAQs](/manuals/desktop/troubleshoo
240240
- Docker Model Runner is now available on x86 Windows machines with NVIDIA GPUs.
241241
- You can now [push models](/manuals/ai/model-runner.md#push-a-model-to-docker-hub) to Docker Hub with Docker Model Runner.
242242
- Added support for Docker Model Runner's model management and chat interface in Docker Desktop for Mac and Windows (on hardware supporting Docker Model Runner). Users can now view, interact with, and manage local AI models through a new dedicated interface.
243-
- [Docker Compose](/manuals/ai/compose/model-runner.md) and Testcontainers [Java](https://java.testcontainers.org/modules/docker_model_runner/) and [Go](https://golang.testcontainers.org/modules/dockermodelrunner/) now support Docker Model Runner.
243+
- [Docker Compose](/manuals/ai/compose/models-and-compose.md) and Testcontainers [Java](https://java.testcontainers.org/modules/docker_model_runner/) and [Go](https://golang.testcontainers.org/modules/dockermodelrunner/) now support Docker Model Runner.
244244
- Introducing Docker Desktop in the [Microsoft App Store](https://apps.microsoft.com/detail/xp8cbj40xlbwkx?hl=en-GB&gl=GB).
245245

246246
### Upgrades

0 commit comments

Comments
 (0)