-
Notifications
You must be signed in to change notification settings - Fork 849
Description
Is your feature request related to a problem? Please describe.
In high-performance or data-intensive workloads on Amazon EKS, it’s common for multiple co-located pods on the same node to require access to a shared dataset—for example, files cached from Amazon S3 or data pre-populated on an EBS volume prior to node launch.
However, Kubernetes does not natively support exposing a single node-local volume—such as a pre-attached EBS volume—to multiple pods on the same node without relying on non-portable constructs like hostPath, using local PersistentVolumes with a single-node scheduling constraint, or building a custom controller to orchestrate volume sharing.
While the existing EBS CSI driver enables sharing a volume among multiple pods on a single same node, it is not designed to scale this pattern cluster-wide to ensure pods on each node can share their respective node-local volumes seamlessly.
Describe the solution you'd like in detail
I am seeking support for safely mounting a shared, node-local volume into multiple pods running on the same node. This capability could be enabled by enhancing the existing EBS CSI Driver’s static provisioning functionality. Specifically, rather than requiring the EBS volume ID to be hardcoded in the PersistentVolume definition (e.g., volumeHandle: vol-03c604538dd7d2f41), the CSI driver could support a node-resolved identifier such as the device name (e.g., deviceName: /dev/xvdbb). This approach would allow the driver to identify the existing EBS volume already attached to the node where the pod is running.
With this enhancement, each pod would be able to share its respective node’s EBS volume independently of volumes on other nodes, enabling safe, scalable sharing of node-local volumes across multiple pods on the same node.
Existing static provisioning PV
apiVersion: v1
kind: PersistentVolume
metadata:
name: test-pv
spec:
# ... other spec fields ...
csi:
driver: ebs.csi.aws.com
volumeHandle: vol-03c604538dd7d2f41
# ... other CSI fields ...Suggested static provisioning PV
apiVersion: v1
kind: PersistentVolume
metadata:
name: test-pv
spec:
# ... other spec fields ...
csi:
driver: ebs.csi.aws.com
# Hardcoded value signalling the CSI driver to look up a locally attached volume on the node
volumeHandle: local-ebs-volume
volumeAttributes:
# The device name or any other identifier for the CSI driver to use to find a locally attached volume on the node
deviceName: /dev/xvdbb
# ... other CSI fields ...This enhancement would:
- Allow a single PersistentVolume and PVC to be defined cluster-wide but scoped to a node.
- Enable multiple pods on the same node to mount the same volume concurrently.
- Each node has it’s own EBS volume and all pods running on the same node sharing the node’s attached EBS volume.
- Leverage the existing EBS CSI plugin with minimal changes, focusing only on the static provisioning and node publishing path.
Describe alternatives you've considered
-
Dynamic Provisioning per Pod (Standard EBS CSI Driver Flow): Each pod receives its own dynamically provisioned EBS volume, managed by the CSI driver. The problems are:
- Pods cannot natively share data across volumes.
- Not cost-effective due to one EBS volume per pod.
- Hits the per-instance EBS volume attachment limits in high-density pod scenarios.
- Slower startup times due to volume provisioning and attachment delays.
-
Custom Control Loop That Directly Creates Pods on Specific Nodes: This approach involves building a custom controller that bypasses Kubernetes’ native scheduler by directly creating pods with a specified nodeName, effectively controlling where each pod runs. The problems are:
- Incompatible with workloads that rely on Kubernetes’ native scheduling to optimize placement across diverse node types.
- EKS environments often use a wide range of EC2 instance types with varying CPU, memory, and ephemeral storage configurations—bypassing the scheduler forfeits the ability to match pod resource requirements to the most suitable node.
- Adds operational complexity and reduces portability across environments.