AdaNomad AdaNomad

GPU Compute Technology Solutions

Does everyone run the NVIDIA GPU Operator?

The NVIDIA GPU Operator is a useful way to manage an NVIDIA Kubernetes stack, but each accelerator needs its own runtime plan.

Jason Sun
2 min read

I was repricing GPU boxes when I caught myself asking: does everyone run the NVIDIA GPU Operator? No.

For NVIDIA GPUs in Kubernetes, I would start there. But it does not manage every accelerator I might own.

The question matters because “we have GPUs” is not a deployment plan. A cluster must discover the devices, expose them to workloads, and keep the host software compatible with those workloads.

What the operator buys me

The NVIDIA GPU Operator manages drivers, the container toolkit, the Kubernetes device plugin, node labels, and monitoring. On supported hardware, it can also manage Multi-Instance GPU partitions.

That is a useful bundle. I would use it when I want one supported way to manage that stack across a cluster.

I can also deploy the NVIDIA device plugin directly when drivers and the container runtime are already handled. Running NVIDIA GPUs in Kubernetes does not prove the operator is installed.

What happens outside that stack

AMD has ROCm and its own GPU Operator. Its data center accelerators have a real Kubernetes path. For a workstation or consumer card, I check the exact ROCm and operator support matrix before promising it will work.

Intel Gaudi has a Kubernetes device plugin. It is another path for accelerator workloads, with its own software requirements. I would not pretend a CUDA container will run there unchanged.

Apple Silicon is different again. I can use Metal and MLX for local inference, but I do not treat a Mac as a sliceable Kubernetes GPU node. A local machine can still be useful compute without fitting the cluster GPU model.

The on-prem question I ask instead

For hardware I own, I make three checks before adding a machine to a pool:

  1. Identify the accelerator. Record the device, host driver, and supported runtime.
  2. Run a real workload. Start the model and measure one representative request on that machine.
  3. Advertise only what works. Give the node capability tags that match the tested runtime and model.

The gotcha: a GPU can appear as a Kubernetes resource while the workload still fails to load its runtime libraries. I test the container, not just device discovery.

So no, everyone does not run the NVIDIA GPU Operator. It solves an important NVIDIA cluster problem. The broader job is matching each workload to the hardware and software that can run it.

If you are deciding which boxes belong in a compute pool, book a call with me. I can help check the runtime plan before you commit to the hardware.