From Device Plugin to DRA, I systematically reviewed the evolution route of Kubernetes device resource management.

From Device Plugin to DRA, I systematically reviewed the evolution route of Kubernetes device resource management. Hardcore post from AI Infra! Collect it first and then take a closer look👀 Many people who come into contact with DRA for the first time will think that it is just a few new CRDs: ResourceClaim, DeviceClass, ResourceSlice. In fact, these APIs correspond to a whole pitfall that AI Infra has stepped on in the past few years. The original Kubernetes didn’t understand GPUs at all. The Device Plugin solves the first problem: making the GPU visible to Kubernetes. Pod can finally write: resources: limits: 1 But problems soon arise. Scheduler only knows that you want 1 GPU. It does not know: * How much video memory is left * Whether the GPU Core is idle * NVLink topology * MIG slicing * NUMA relationships * Multi-card affinity For today's large model training and inference clusters, this information is often more important than "a few cards". So the second phase of plans began to appear in the community. Projects such as HAMi, Volcano vGPU, and Koordinator GPU Sharing have emerged one after another. The core idea at this stage is to add an AI Resource Layer to Kubernetes. GPU begins to have: * Video memory quota * Computing power quota * Topology awareness * GPU sharing * GPU isolation Many internal AI platforms of enterprises are actually developed on this layer. But these solutions have one characteristic in common: they all belong to plug-in mode. Scheduler Extender, Webhook, Node Annotation, Custom Resource... The functions are becoming more and more powerful, and the system is becoming more and more complex. So the Kubernetes community began to think about a question: Why can't device resources become Kubernetes native objects like storage? PVC has StorageClass PVC Why can't PV devices also have dynamic resources? DeviceClass ResourceClaim ResourceSlice Then DRA appeared. What DRA (Dynamic Resource Allocation) does is simple: upgrade device resources to Kubernetes first-class citizens. From now on: The scheduler can understand the device. Autoscaler understands devices. Quota can understand the device. Future Billing, Reservation, and Inference Scheduling can also be built on the unified resource model. From the perspective of AI Infra: Device Plugin solves the device access problem. HAMi solves the device sharing problem. DRA is addressing equipment resource standardization issues. I increasingly feel that the base of AI Resource OS in the next few years will most likely be built on DRA. Recommended reading: ① Kubernetes DRA official documentation ② Kubernetes

34: DRA GA ③ Google Cloud in-depth interpretation ④ Azure AKS practice ⑤ NVIDIA DRA Driver ⑥ NVIDIA DRA Driver source code ⑦ Kubernetes DRA Tutorial Understand DRA, and then look back at HAMi, Volcano, and Kueue, it will be much clearer.

Visual summary

Article argument map

Generated from the post's content graph

Mermaid outline

flowchart LR
  format-article["article feature"]
  topic-memory["memory"]
  topic-inference["inference"]
  topic-retrieval["retrieval"]
  capability-technical-writing["technical writing"]
  format-article -->|covers| topic-memory
  format-article -->|covers| topic-inference
  format-article -->|covers| topic-retrieval
  format-article -->|signals| capability-technical-writing

Visual structure

Essay structure map

Built from summary and key paragraph positions

Mermaid outline

flowchart LR
  thesis["From Device Plugin to DRA, I systematically reviewed the evolution route of Kubernetes device resource mana..."]
  signal["From Device Plugin to DRA, I systematically reviewed the evolution route of Kubernetes device resource mana..."]
  operator["1.34: DRA GA ③ Google Cloud in-depth interpretation ④ Azure AKS practice ⑤ NVIDIA DRA Driver ⑥ NVIDIA DRA D..."]
  implication["1.34: DRA GA ③ Google Cloud in-depth interpretation ④ Azure AKS practice ⑤ NVIDIA DRA Driver ⑥ NVIDIA DRA D..."]
  thesis -->|frames| signal
  signal -->|develops| operator
  operator -->|lands in| implication

Source: View the original post