Skip to content

engine: containerd #23198

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 17 commits into
base: moby29
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .vscode/docker.code-snippets
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,29 @@
"body": ["{{< button url=\"$1\" text=\"$2\" >}}"],
"description": "Insert a Hugo button",
},
"Insert a Summary Bar": {
"prefix": ["summary-bar"],
"body": [
"{{< summary-bar feature_name=\"$1\" ${2| ,text=\"some text\"|} >}}",
"",
"$3",
],
"description": "Insert a Hugo summary bar with a feature name and content. Define your summary key in data/summary.yaml.",
},
"Insert Title": {
"prefix": ["insert title", "frontmatter", "title"],
"body": [
"---",
"title: $1",
"weight: $2",
"description: $3",
"keywords: $4",
"aliases:",
" - $5",
"---"
],
"description": "Insert a frontmatter block with title, weight, description, keywords, and aliases."
}
"Insert Visual Studio Code": {
"prefix": ["vscode", "vs"],
"body": ["Visual Studio Code"],
Expand Down
3 changes: 2 additions & 1 deletion _vale/config/vocabularies/Docker/accept.txt
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,7 @@ Qualcomm
Quickview
rebalance
reimplement
repull
rollback
rootful
runc
Expand All @@ -167,7 +168,6 @@ S3
scrollable
SELinux
Slack
snapshotters?
Snyk
Solr
SonarQube
Expand Down Expand Up @@ -253,6 +253,7 @@ Zsh
[Rr]untimes?
[Ss]andbox(ed)?
[Ss]eccomp
[sS]napshotters?
[Ss]ubmounts?
[Ss]ubnet
[Ss]ubpaths?
Expand Down
3 changes: 3 additions & 0 deletions assets/css/components.css
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,9 @@
.summary-bar {
@apply my-1 mt-4 flex flex-col rounded-sm border-1 border-gray-100 bg-gray-50 p-4 dark:border-gray-800 dark:bg-gray-900;
}
.summary-bar:has(.deprecated) {
@apply bg-red-100 dark:bg-red-950;
}

.tabs {
@apply bg-blue/2 rounded-sm p-2;
Expand Down
1 change: 1 addition & 0 deletions content/manuals/admin/company/owners.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ entire company and can manage settings that apply to all organizations under
that company. They also have the same access rights as organization owners but
don’t need to be members of any individual organization.


> [!IMPORTANT]
>
> Company owners do not occupy a seat unless one of the following is true:
Expand Down
11 changes: 11 additions & 0 deletions content/manuals/engine/storage/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,17 @@ container if another process needs it.
The writable layer is unique per container. You can't easily extract the data
from the writeable layer to the host, or to another container.

> [!IMPORTANT]
> Starting with Docker Engine v29, Docker uses
> [containerd](./containerd.md) for managing container storage and images.

## Image storage

Docker offers two implementations for image storage:

- Default/current implementation: [Containerd](./containerd.md)
- Legacy/deprecated implementation: [Storage drivers](./drivers/_index.md)

## Storage mount options

Docker supports the following types of storage mounts for storing data outside
Expand Down
60 changes: 37 additions & 23 deletions content/manuals/engine/storage/containerd.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,49 +8,63 @@ aliases:
- /storage/containerd/
---

{{< summary-bar feature_name="containerd" >}}
Starting with Docker Engine v29, Docker uses [`containerd`](https://containerd.io/)
for managing container storage and images. `containerd` is the industry-standard container runtime.

containerd, the industry-standard container runtime, uses snapshotters instead
of the classic storage drivers for storing image and container data.
While the `overlay2` driver still remains the default driver for Docker Engine,
you can opt in to using containerd snapshotters as an experimental feature.
## Main benefits

To learn more about the containerd image store and its benefits, refer to
containerd offers the following benefits:

- Shared maintenance: containerd is an open-source project maintained by a large community.
- Customizability: use [snapshotters](snapshotters.md)
with unique characteristics, such as:
- [stargz](https://github.com/containerd/stargz-snapshotter) for lazy-pulling images on container
startup.
- [nydus](https://github.com/containerd/nydus-snapshotter) or [dragonfly](https://github.com/dragonflyoss/nydus) for peer-to-peer image distribution.
- Portability: containerd is lighter and more modular.
- Multi-platform support: Build multi-platform images and images with attestations.
- WebAssembly: Ability to run Wasm containers.

For more information about the containerd image store and its benefits, refer to
[containerd image store on Docker Desktop](/manuals/desktop/features/containerd.md).

## Enable containerd image store on Docker Engine
## Migrate to containerd image store on Docker Engine
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like I was a little fast on the gun here. Sorry about that. The containerd migration feature will be experimental in v29 and can be enabled with the following daemon configuration.

{
    "features": {
        "containerd-migration": true
    }
}

IDK if we need to clarify this is useful for users upgrading in place. Fresh installs wouldn't find it very useful. WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the section. As for your question, I would say "less is more", people that start fresh will surely know they have nothing to migrate so would skip this section?


Switching to containerd snapshotters causes you to temporarily lose images and
containers created using the classic storage drivers.
Those resources still exist on your filesystem, and you can retrieve them by
turning off the containerd snapshotters feature.
{{< summary-bar feature_name="Containerd migration" >}}

The following steps explain how to enable the containerd snapshotters feature.
You can enable the auto-migration feature for containerd snapshotters once you
update to Docker Engine v29. The migration mechanism handles overlay and vfs
images. To enable it:

1. Add the following configuration to your `/etc/docker/daemon.json`
configuration file:

```json
{
"features": {
"containerd-snapshotter": true
"containerd-migration": true
}
}
```

2. Save the file.
3. Restart the daemon for the changes to take effect.
1. Save the file.

1. Restart the daemon for the changes to take effect.

Switching to containerd snapshotters causes you to temporarily lose images and
containers created using the classic storage drivers.
If you use other graph storage, repull or rebuild your images.

1. To display which driver you are using, run:

```console
$ sudo systemctl restart docker
$ docker info -f '{{ .DriverStatus }}'
```

After restarting the daemon, running `docker info` shows that you're using
containerd snapshotter storage drivers.
> [!TIP]
> Those resources still exist on your filesystem, and you can retrieve them by
> [turning off the containerd snapshotters feature](./drivers/_index.md#enable-legacy-storage-drivers).

```console
$ docker info -f '{{ .DriverStatus }}'
[[driver-type io.containerd.snapshotter.v1]]
```
## Related pages

Docker Engine uses the `overlayfs` containerd snapshotter by default.
- [Legacy storage drivers](drivers/_index.md)
66 changes: 50 additions & 16 deletions content/manuals/engine/storage/drivers/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,15 @@
description: Learn the technologies that support storage drivers.
keywords: container, storage, driver, btrfs, overlayfs, vfs, zfs
title: Storage drivers
weight: 40
aliases:
- /storage/storagedriver/imagesandcontainers/
- /storage/storagedriver/
- /engine/userguide/storagedriver/imagesandcontainers/
---

{{< summary-bar feature_name="StorageDrivers"
text="Use [containerd](../containerd.md) instead.">}}

To use storage drivers effectively, it's important to know how Docker builds and
stores images, and how these images are used by containers. You can use this
information to make informed choices about the best way to persist data from
Expand Down Expand Up @@ -170,7 +172,7 @@ docker.io/library/ubuntu:22.04

Each of these layers is stored in its own directory inside the Docker host's
local storage area. To examine the layers on the filesystem, list the contents
of `/var/lib/docker/<storage-driver>`. This example uses the `overlay2`
of `/var/lib/docker/<storage-driver>`. This example uses the `overlay2`
storage driver:

```console
Expand Down Expand Up @@ -326,7 +328,7 @@ layers are the same.
Before BuildKit, the "classic" builder would produce a new "intermediate"
image for each step for caching purposes, and the `IMAGE` column would show
the ID of that image.

BuildKit uses its own caching mechanism, and no longer requires intermediate
images for caching. Refer to [BuildKit](/manuals/build/buildkit/_index.md)
to learn more about other enhancements made in BuildKit.
Expand All @@ -343,7 +345,7 @@ layers are the same.
"sha256:07b4a9068b6af337e8b8f1f1dae3dd14185b2c0003a9a1f0a6fd2587495b204a"
]
```

```console
$ docker image inspect --format "{{json .RootFS.Layers}}" acme/my-final-image:1.0
[
Expand All @@ -363,7 +365,7 @@ layers are the same.
> [!TIP]
>
> Format output of Docker commands with the `--format` option.
>
>
> The examples above use the `docker image inspect` command with the `--format`
> option to view the layer IDs, formatted as a JSON array. The `--format`
> option on Docker commands can be a powerful feature that allows you to
Expand Down Expand Up @@ -412,7 +414,7 @@ in a `copy_up` operation, therefore duplicating the file to the writable layer.
> applications, for example write-intensive databases, are known to be
> problematic particularly when pre-existing data exists in the read-only
> layer.
>
>
> Instead, use Docker volumes, which are independent of the running container,
> and designed to be efficient for I/O. In addition, volumes can be shared
> among containers and don't increase the size of your container's writable
Expand Down Expand Up @@ -449,7 +451,7 @@ examines how much room they take up.
2. Run the `docker ps` command with the `--size` option to verify the 5 containers
are running, and to see each container's size.


```console
$ docker ps --size --format "table {{.ID}}\t{{.Image}}\t{{.Names}}\t{{.Size}}"

Expand All @@ -460,35 +462,35 @@ examines how much room they take up.
a5ff32e2b551 acme/my-final-image:1.0 my_container_2 0B (virtual 7.75MB)
40ebdd763416 acme/my-final-image:1.0 my_container_1 0B (virtual 7.75MB)
```

The output above shows that all containers share the image's read-only layers
(7.75MB), but no data was written to the container's filesystem, so no additional
storage is used for the containers.

{{< accordion title="Advanced: metadata and logs storage used for containers" >}}

> [!NOTE]
>
> This step requires a Linux machine, and doesn't work on Docker Desktop, as
> it requires access to the Docker Daemon's file storage.

While the output of `docker ps` provides you information about disk space
consumed by a container's writable layer, it doesn't include information
about metadata and log-files stored for each container.

More details can be obtained by exploring the Docker Daemon's storage
location (`/var/lib/docker` by default).

```console
$ sudo du -sh /var/lib/docker/containers/*

36K /var/lib/docker/containers/3ed3c1a10430e09f253704116965b01ca920202d52f3bf381fbb833b8ae356bc
36K /var/lib/docker/containers/40ebdd7634162eb42bdb1ba76a395095527e9c0aa40348e6c325bd0aa289423c
36K /var/lib/docker/containers/939b3bf9e7ece24bcffec57d974c939da2bdcc6a5077b5459c897c1e2fa37a39
36K /var/lib/docker/containers/a5ff32e2b551168b9498870faf16c9cd0af820edf8a5c157f7b80da59d01a107
36K /var/lib/docker/containers/cddae31c314fbab3f7eabeb9b26733838187abc9a2ed53f97bd5b04cd7984a5a
```

Each of these containers only takes up 36k of space on the filesystem.

{{< /accordion >}}
Expand All @@ -502,13 +504,13 @@ examines how much room they take up.
```console
$ for i in {1..3}; do docker exec my_container_$i sh -c 'printf hello > /out.txt'; done
```

Running the `docker ps` command again afterward shows that those containers
now consume 5 bytes each. This data is unique to each container, and not
shared. The read-only layers of the containers aren't affected, and are still
shared by all containers.


```console
$ docker ps --size --format "table {{.ID}}\t{{.Image}}\t{{.Names}}\t{{.Size}}"

Expand All @@ -533,7 +535,39 @@ work, with one or more virtual disks per virtual machine. The [`vfs` storage](vf
doesn't provide a CoW filesystem or other optimizations. When using this storage
driver, a full copy of the image's data is created for each container.

## Enable Legacy storage drivers

Starting with Docker Engine v29, [containerd](../containerd.md) is enabled by
default. To re-enable the legacy storage drivers instead of using containerd
snapshotters:

1. Add the following configuration to your `/etc/docker/daemon.json`
configuration file:

```json
{
"features": {
"containerd-snapshotter": false
}
}
```

1. Save the file.

1. Restart the daemon for the changes to take effect.

```console
$ sudo systemctl restart docker
```

1. Verify which driver you are using:

```console
$ docker info -f '{{ .DriverStatus }}'
```

## Related information

* [Volumes](../volumes.md)
* [Select a storage driver](select-storage-driver.md)
* [Containerd](../containerd.md)
25 changes: 25 additions & 0 deletions content/manuals/engine/storage/snapshotters.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
title: containerd snapshotters
description: Learn how to use containerd snapshotters in Docker Engine.
keywords: containerd, snapshotters, storage, overlayfs, fuse, docker, engine, filesystem, performance
---

containerd snapshotters are components responsible for managing the filesystem
layers of containers. They provide the mechanism for storing, mounting, and
manipulating container filesystems using different backends such as overlayfs or
fuse.

By abstracting the storage implementation, snapshotters allow Docker Engine to
efficiently manage container images and their writable layers, enabling features
like fast image pulls, efficient storage usage, and support for advanced
filesystems.

For more information, see
the [containerd repository](https://github.com/containerd/containerd/tree/main/docs/snapshotters).

| Snapshotter | Description |
|-----------------------|----------------------------------------------------------------------------------------|
| `overlayfs` (default) | OverlayFS. The containerd implementation of the Docker/Moby `overlay2` storage driver. |
| `native` | Native file copying driver. Akin to Docker/Moby's "vfs" driver. |


6 changes: 4 additions & 2 deletions data/summary.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -133,8 +133,6 @@ Composefile include:
requires: Docker Compose [2.20.0](/manuals/compose/releases/release-notes.md#2200) and later
Compose sbom:
requires: Docker Compose [2.39.0](/manuals/compose/releases/release-notes.md#2390) and later
containerd:
availability: Experimental
Dev Environments:
availability: Beta
Docker Build Cloud:
Expand Down Expand Up @@ -257,3 +255,7 @@ Wasm workloads:
availability: Beta
Wasmtime:
availability: Experimental
StorageDrivers:
availability: Deprecated
Containerd migration:
availability: Experimental
Loading