Sharing the GPU — MIG, MPS, vGPU, and Kubernetes Time-Slicing

Topics We'll Cover

Four sharing models, three isolation tiers, one decision to make per cluster. This deck walks the decision tree from "why share at all?" through to ready-to-paste YAML.

• Why Share a GPU?
• The Four Sharing Models
• MIG — Multi-Instance GPU
• MIG Limits & Gotchas
• MPS — Multi-Process Service
• MIG vs MPS — When to Use Which
• vGPU — Virtualisation with NVIDIA AI Enterprise
• NVIDIA AI Enterprise & Licensing
• Kubernetes — Time-Slicing & GPU Operator
• Putting It Together — Worked Tenancy Patterns
• Interactive: Sharing-Mode Picker

Why Share a GPU?

The single workload-per-GPU model dates from when GPUs had 8–16 GB of VRAM and one model filled the card. A 2026 H100 with 80–96 GB of HBM running a single 7B inference engine is closer to a yacht being used as a kayak.

Cost arithmetic

The Four Sharing Models

NVIDIA offers four largely orthogonal sharing mechanisms. They live at different layers of the stack and can be combined.

Where each one lives in the stack

MIG — Multi-Instance GPU

MIG is the only sharing mode with hardware enforcement. The GPU is physically partitioned at the GPC (Graphics Processing Cluster) and memory-controller boundary. There is no path through silicon for one slice's traffic to perturb another.

What gets partitioned

• SMs — assigned to a GPU Instance (GI); never shared.
• L2 cache — each GI gets a dedicated 1/N slice of L2, including its tag pipeline.
• HBM — dedicated memory regions and dedicated memory controllers. Bandwidth is partitioned, not shared.
• NVDEC / NVENC / NVJPG — partitioned where present.
• A GI further hosts one or more Compute Instances (CI) — CI is where your CUDA context actually runs.

Because controllers are independent, the noisy-neighbour problem that plagues MPS and time-slicing is physically impossible: a runaway memcpy in slice 1 cannot steal HBM bandwidth from slice 2.

H100 80 GB — canonical profile catalogue

Up to 7 instances per H100. Mix profiles freely as long as the total fits (e.g. 3g + 2g + 1g + 1g).

Configuring MIG by hand

# Enable MIG mode (requires no compute in flight; persistent across reboot)
nvidia-smi -mig 1

# Create GPU Instances by profile ID:  9 = 3g.40gb, 14 = 2g.20gb, 19 = 1g.10gb
nvidia-smi mig -cgi 9,14,19,19,19

# Create one Compute Instance per GI (default profile = whole GI)
nvidia-smi mig -cci

# See what you got
nvidia-smi -L
# GPU 0: NVIDIA H100 80GB HBM3 (UUID: GPU-...)
#   MIG 3g.40gb Device 0: (UUID: MIG-...)   ← tenant A sees this as $CUDA_VISIBLE_DEVICES
#   MIG 2g.20gb Device 1: (UUID: MIG-...)
#   MIG 1g.10gb Device 2: ...

Each MIG device shows up as a UUID that workloads pin via CUDA_VISIBLE_DEVICES=MIG-<uuid>. Tenants see what looks like a normal, smaller, dedicated GPU.

MIG Limits & Gotchas

MIG's hardware partitioning is its strength and its straitjacket. The same isolation that prevents noisy neighbours also prevents collaboration.

MPS — Multi-Process Service

MPS is the older mechanism and is fundamentally different: rather than partition the hardware, it multiplexes CUDA contexts in software so multiple processes look like one to the GPU.

How it works

Without MPS, the GPU time-slices at the context level: kernel from A finishes, context switch, kernel from B finishes, switch back. Context switches on a GPU are not free. MPS folds all clients into a single context so kernels from any client can co-execute on different SMs concurrently.

Wins

• Lower latency than time-slicing — no CUDA context switch between clients.
• Concurrent kernel execution — small kernels from different clients fill the SM array. Inference servers with many small models love this.
• Universal availability — user-space MPS exists on virtually every CUDA GPU; Volta and later add hardware-accelerated MPS with per-client priority and SM-percentage limits. Includes consumer cards where MIG isn't an option.

Loses

• No memory isolation — one allocation lives in the same context as every other. One bug OOMs all tenants.
• No fair-share by default — a tenant launching million-block kernels can starve a tenant launching small ones.
• Crude limits — CUDA_MPS_ACTIVE_THREAD_PERCENTAGE caps the fraction of SMs a client may dispatch onto, but it's a soft cap, per-launch, not enforced at SM level.
• Single failure domain — the daemon crashing kills every client.

# Start the daemon (typically as a systemd unit)
export CUDA_VISIBLE_DEVICES=0
export CUDA_MPS_PIPE_DIRECTORY=/tmp/nvidia-mps
export CUDA_MPS_LOG_DIRECTORY=/var/log/nvidia-mps
nvidia-cuda-mps-control -d

# Cap a client at 30% of SMs (best-effort)
export CUDA_MPS_ACTIVE_THREAD_PERCENTAGE=30
./my_inference_server

MIG vs MPS — When to Use Which

The decision usually compresses to one question: do these workloads trust each other?

Use both, layered

The strongest pattern in production is MIG for tenant isolation, MPS within each MIG slice for many processes per tenant. Tenant A gets a 2g.20gb slice. Inside the slice, Triton runs MPS so its 12 model backends co-execute. Tenant B's 1g.10gb slice is similarly arranged but cannot perturb A's. You get hardware isolation between strangers and software co-execution among friends.

vGPU — Virtualisation with NVIDIA AI Enterprise

vGPU is NVIDIA's enterprise virtualisation product, formerly branded GRID. It exposes one physical GPU to a hypervisor as N virtual GPUs, each of which a VM consumes as a normal device. Required where the isolation boundary is the VM, not the process.

Hypervisors supported

Two backing modes

Licensing & pricing

vGPU is licensed per concurrent vGPU, sold as named editions:

Discounts in volume are deep; single-seat list prices rarely apply to enterprise deals.

NVIDIA AI Enterprise & Licensing

The "NVIDIA AI Enterprise" (NVAIE) product is the 2026 packaging of vGPU plus a large software estate. If you want vGPU at all on compute cards, you are buying NVAIE.

What's bundled

• vGPU manager — the host driver enabling everything in slide 07.
• Validated CUDA images — certified container images for RHEL/Ubuntu against tested driver/CUDA pairs.
• NeMo — LLM training and customisation framework.
• NIM (NVIDIA Inference Microservices) — pre-packaged inference containers for popular open-weight models.
• Triton Inference Server — multi-framework inference runtime.
• Base Command — cluster orchestration / job scheduling.
• Enterprise support — SLA-backed driver and CUDA support; the part you're really paying for.

Price shape

List price is roughly $4,500 per GPU per year, often steeply discounted in large hyperscaler or enterprise commitments. Multi-year terms (3 / 5 yr) and CSP perpetual licences exist.

Kubernetes — Time-Slicing & GPU Operator

Kubernetes' default device plugin model is unforgiving: one GPU = one pod. Anything finer-grained needs the NVIDIA GPU Operator, which manages drivers, plugins, MIG configuration and time-slicing as a coherent stack.

Default behaviour

resources:
  limits:
    nvidia.com/gpu: 1   # one whole GPU. Nothing else can claim it.

Time-slicing config

Configured in the GPU Operator's ClusterPolicy: replicas: N tells the device plugin to advertise the same GPU as N "shadows". N pods can claim it concurrently; the driver round-robins their kernel launches.

apiVersion: nvidia.com/v1
kind: ClusterPolicy
spec:
  devicePlugin:
    config:
      name: time-slicing-config
  sharing:
    timeSlicing:
      resources:
        - name: nvidia.com/gpu
          replicas: 4      # 4 pods may share each GPU

• Loose isolation. No memory cap; one pod's cudaMalloc reduces the pool every other pod sees.
• Round-robin scheduling. Kernel launches interleave at GPU context-switch granularity.
• Useful for dev / test pools, JupyterHub, CI runners — bursty workloads where collisions are rare and recoverable.
• Not for production multi-tenant. Same OOM caveat as MPS.

Production multi-tenant — MIG-aware GPU Operator

For real isolation in K8s, have the GPU Operator configure MIG and surface each slice as its own resource type. Pods request slices like any other typed resource.

resources:
  limits:
    nvidia.com/gpu.product: H100-PCIE-80GB-MIG-1g.10gb
    nvidia.com/gpu: 1

# Operator labels nodes with the available profiles per GPU,
# scheduler matches pod request to a free slice on a labelled node.

Putting It Together — Worked Tenancy Patterns

Three real-world configurations spanning the trust spectrum.

Heuristic for picking the right pattern

1. What is the trust boundary? Process → MPS or time-slicing. VM → vGPU. Tenant → MIG.
2. Is the GPU MIG-capable? If yes and trust boundary > process, default to MIG.
3. Is the workload bursty and small? Time-slicing or MPS will pack better.
4. Do you need a VM? Then vGPU; you're paying for NVAIE either way.
5. Big sharded model? None of the above — that's TP across whole GPUs with NVLink.

Interactive: Sharing-Mode Picker

Pick a GPU class, a workload mix, and an isolation requirement. The picker recommends a sharing mode, estimates fan-out, flags licensing, and emits the right command or YAML snippet.