Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -718,6 +718,12 @@ extender.
}
```

#### Extended resources allocation by DRA
Extended resources allocation by DRA allows cluster administrators to specify an `extendedResourceName`
in `DeviceClass`, then the devices matching the `DeviceClass` can be requested from a pod's extended
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(nit) we prefer not to put backticks around API kinds, such as DeviceClass

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#51815 PTAL

resource requests. Read more about
[Extended Resource allocation by DRA](/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#extended-resource).

### Consuming extended resources

Users can consume extended resources in Pod specs like CPU and memory.
Expand Down Expand Up @@ -934,3 +940,4 @@ memory limit (and possibly request) for that container.
* Read about [project quotas](https://www.linux.org/docs/man8/xfs_quota.html) in XFS
* Read more about the [kube-scheduler configuration reference (v1)](/docs/reference/config-api/kube-scheduler-config.v1/)
* Read more about [Quality of Service classes for Pods](/docs/concepts/workloads/pods/pod-qos/)
* Read more about [Extended Resource allocation by DRA](/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#extended-resource)
Original file line number Diff line number Diff line change
Expand Up @@ -509,3 +509,4 @@ Here are some examples of device plugin implementations:
* Learn about the [Topology Manager](/docs/tasks/administer-cluster/topology-manager/)
* Read about using [hardware acceleration for TLS ingress](/blog/2019/04/24/hardware-accelerated-ssl/tls-termination-in-ingress-controllers-using-kubernetes-device-plugins-and-runtimeclass/)
with Kubernetes
* Read more about [Extended Resource allocation by DRA](/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#extended-resource)
Original file line number Diff line number Diff line change
Expand Up @@ -424,6 +424,46 @@ create ResourceClaim or ResourceClaimTemplate objects in namespaces labeled with
`adminAccess` field. This ensures that non-admin users cannot misuse the
feature.

### Extended resource allocation by DRA {#extended-resource}

{{< feature-state feature_gate_name="DRAExtendedResource" >}}

You can provide an extended resource name for a DeviceClass. The scheduler will then
select the devices matching the class for the extended resource requests. This allows
users to continue using extended resource requests in a pod to request either
extended resources provided by device plugin, or DRA devices. The same extended
resource can be provided either by device plugin, or DRA on one single cluster node.
The same extended resource can be provided by device plugin on some nodes, and
DRA on other nodes in the same cluster.

In the example below, the DeviceClass is given an extendedResourceName `example.com/gpu`.
If a pod requested for the extended resource `example.com/gpu: 2`, it can be scheduled to
a node with two or more devices matching the DeviceClass.

```yaml
apiVersion: resource.k8s.io/v1
kind: DeviceClass
metadata:
name: gpu.example.com
spec:
selectors:
- cel:
expression: device.driver == 'gpu.example.com' && device.attributes['gpu.example.com'].type
== 'gpu'
extendedResourceName: example.com/gpu
```

In addition, users can use a special extended resource to allocate devices without
having to explicitly create a ResourceClaim. Using the extended resource name
prefix `deviceclass.resource.kubernetes.io/` and the DeviceClass name. This works
for any DeviceClass, even if it does not specify the an extended resource name.
The resulting ResourceClaim will contain a request for an `ExactCount` of the
specified number of devices of that DeviceClass.

Extended resource allocation by DRA is an *alpha feature* and only enabled when the
`DRAExtendedResource` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
is enabled in the kube-apiserver, kube-scheduler, and kubelet.

### Prioritized list {#prioritized-list}

{{< feature-state feature_gate_name="DRAPrioritizedList" >}}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
title: DRAExtendedResource
content_type: feature_gate
_build:
list: never
render: false

stages:
- stage: alpha
defaultValue: false
fromVersion: "1.34"
---
Enables support for the [Extended Resource allocation by DRA](/docs/concepts/configuration/manage-resources-containers/#extended-resources-allocation-by-dra) feature.
It makes it possible to specify an extended resource name in a DeviceClass.

This feature gate has no effect unless the `DynamicResourceAllocation` feature gate is enabled.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should revise the DynamicResourceAllocation file so that people know to update this one when they lock DynamicResourceAllocation to enabled.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DynamicResourceAllocation file already has the following, what else do you like to add?

Extended resource allocation by DRA is an alpha feature and only enabled when the
DRAExtendedResource feature gate
is enabled in the kube-apiserver, kube-scheduler, and kubelet.

Original file line number Diff line number Diff line change
Expand Up @@ -188,8 +188,10 @@ kubectl describe node <your-node-name> | grep dongle
### For application developers

- [Assign Extended Resources to a Container](/docs/tasks/configure-pod-container/extended-resource/)
- [Extended Resource allocation by DRA](/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#extended-resource)

### For cluster administrators

- [Configure Minimum and Maximum Memory Constraints for a Namespace](/docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace/)
- [Configure Minimum and Maximum CPU Constraints for a Namespace](/docs/tasks/administer-cluster/manage-resources/cpu-constraint-namespace/)
- [Extended Resource allocation by DRA](/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#extended-resource)