Partitioning GPU instances using MIG

Multi-Instance GPU (MIG) allows you to partition a physical GPU into multiple isolated instances, each with dedicated memory and compute resources. This is particularly useful when you need to run multiple workloads that don’t require a full GPU.

Prerequisites

A GPU instance with a MIG-capable GPU. On Exoscale, the following GPU types support MIG:
- GPU A30
- GPU RTX Pro 6000
A supported NVIDIA datacenter driver installed in the guest (R460 or later)

Install NVIDIA Driver

Install a supported NVIDIA datacenter driver in the guest. For MIG support, you need a driver that supports MIG (R460 or later). We recommend using the server driver variant for stability:

# Add NVIDIA repository
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
    gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

# Install driver (requires reboot)
sudo apt update
sudo apt install -y nvidia-driver-570-server
sudo reboot

Check MIG Capability

Verify that your GPU supports MIG:

nvidia-smi -L
nvidia-smi -q | less

Look for MIG capability in the output. If your GPU supports MIG, you’ll see MIG devices listed.

Enable MIG Mode

Enable MIG mode on the GPU:

sudo nvidia-smi -i 0 -mig 1

Replace 0 with the appropriate GPU index if you have multiple GPUs.

Create GPU Partitions

List available MIG profiles:

sudo nvidia-smi mig -lgip

Create a GPU partition using the profile ID:

sudo nvidia-smi mig --create-gpu-instance <profile_id> --default-compute-instance -i 0

Replace <profile_id> with the ID of the desired MIG profile from the previous step. The --default-compute-instance flag creates a compute instance along with the GPU instance.

Verify MIG Devices

Confirm the created MIG devices:

nvidia-smi -L

You should see multiple MIG devices listed, each with its own UUID.

Run Workloads on MIG Devices

Using CUDA

Set the CUDA_VISIBLE_DEVICES environment variable to target a specific MIG device:

export CUDA_VISIBLE_DEVICES=MIG-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

Replace the UUID with the MIG device you want to use. You can get the UUID from the nvidia-smi -L output.

Using Docker

First, install Docker and the NVIDIA container toolkit:

# Install Docker
curl -fsSL https://get.docker.com | sh

# Add NVIDIA container toolkit
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
    gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt update
sudo apt install -y nvidia-container-toolkit
sudo systemctl restart docker

You can run Docker containers against specific MIG devices using the NVIDIA runtime:

docker run --gpus ' MIG-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' nvidia/cuda:12.9.1-base-ubuntu24.04 nvidia-smi

To run two Docker containers each using a different MIG partition:

# Get MIG device UUIDs
MIG_UUIDS=$(nvidia-smi -L | grep MIG | awk -F'MIG-' '{print "MIG-" $2}' | awk -F')' '{print $1}')

# Extract individual UUIDs
MIG1=$(echo $MIG_UUIDS | cut -d' ' -f1)
MIG2=$(echo $MIG_UUIDS | cut -d' ' -f2)

# Run container on first MIG device
docker run --gpus "$MIG1" -it --name container1 nvidia/cuda:12.9.1-base-ubuntu24.04 nvidia-smi

# Run container on second MIG device
docker run --gpus "$MIG2" -it --name container2 nvidia/cuda:12.9.1-base-ubuntu24.04 nvidia-smi

Considerations

MIG partitions are fixed in size and cannot be resized after creation
Each MIG instance has dedicated memory that cannot be oversubscribed
Not all GPUs support MIG - check NVIDIA’s documentation for supported hardware
MIG mode must be enabled before creating partitions and cannot be disabled without rebooting

Last updated on March 30, 2026