# Use GPU MIG Partitioning

[Multi-Instance GPU (MIG)](https://www.nvidia.com/en-us/technologies/multi-instance-gpu/) allows you to partition a physical GPU into multiple isolated instances, each with dedicated memory and compute resources. This is particularly useful when you need to run multiple workloads that don't require a full GPU.

## Prerequisites

- A GPU instance with a MIG-capable GPU. On Exoscale, the following GPU types support MIG:
  - [GPU A30](https://www.exoscale.com/gpu/a30/)
  - [GPU RTX Pro 6000](https://www.exoscale.com/gpu/rtx6000/)
- A supported NVIDIA datacenter driver installed in the guest (R460 or later)

## Install NVIDIA Driver

Install a supported NVIDIA datacenter driver in the guest. For MIG support, you need a driver that supports MIG (R460 or later). We recommend using the server driver variant for stability:

```shell
# Add NVIDIA repository
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
    gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

# Install driver (requires reboot)
sudo apt update
sudo apt install -y nvidia-driver-570-server
sudo reboot
```

## Check MIG Capability

Verify that your GPU supports MIG:

```shell
nvidia-smi -L
nvidia-smi -q | less
```

Look for MIG capability in the output. If your GPU supports MIG, you'll see MIG devices listed.

## Enable MIG Mode

Enable MIG mode on the GPU:

```shell
sudo nvidia-smi -i 0 -mig 1
```

Replace `0` with the appropriate GPU index if you have multiple GPUs.

## Create GPU Partitions

List available MIG profiles:

```shell
sudo nvidia-smi mig -lgip
```

Create a GPU partition using the profile ID:

```shell
sudo nvidia-smi mig --create-gpu-instance <profile_id> --default-compute-instance -i 0
```

Replace `<profile_id>` with the ID of the desired MIG profile from the previous step. The `--default-compute-instance` flag creates a compute instance along with the GPU instance.

> On GPU A30, MIG configurations persist beyond the lifecycle of the instance.  
> Make sure to [clean up the configuration](#clean-up-mig-configuration-applicable-for-a30-only) before stopping or deleting the VM to avoid impacting future workloads.

## Verify MIG Devices

Confirm the created MIG devices:

```shell
nvidia-smi -L
```

You should see multiple MIG devices listed, each with its own UUID.

## Clean Up MIG Configuration (applicable for A30 only)

On GPU A30, MIG configurations persist on the GPU even after a virtual machine is stopped or deleted.

This means that if a GPU is reused by another instance, previously created MIG partitions may still be present and affect expected behavior.

To avoid conflicts or unexpected errors, it is recommended to remove any MIG configuration before stopping or deleting the instance. Also apply this command in case the GPU instance started with an unwanted MIG configuration to access the full GPU A30 performance.

Example:

```shell
sudo nvidia-smi mig -i 0 -dgi
```

## Run Workloads on MIG Devices

### Using CUDA

Set the `CUDA_VISIBLE_DEVICES` environment variable to target a specific MIG device:

```shell
export CUDA_VISIBLE_DEVICES=MIG-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
```

Replace the UUID with the MIG device you want to use. You can get the UUID from the `nvidia-smi -L` output.

### Using Docker

First, install Docker and the NVIDIA container toolkit:

```shell
# Install Docker
curl -fsSL https://get.docker.com | sh

# Add NVIDIA container toolkit
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
    gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt update
sudo apt install -y nvidia-container-toolkit
sudo systemctl restart docker
```

You can run Docker containers against specific MIG devices using the NVIDIA runtime:

```shell
docker run --gpus ' MIG-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' nvidia/cuda:12.9.1-base-ubuntu24.04 nvidia-smi
```

To run two Docker containers each using a different MIG partition:

```shell
# Get MIG device UUIDs
MIG_UUIDS=$(nvidia-smi -L | grep MIG | awk -F'MIG-' '{print "MIG-" $2}' | awk -F')' '{print $1}')

# Extract individual UUIDs
MIG1=$(echo $MIG_UUIDS | cut -d' ' -f1)
MIG2=$(echo $MIG_UUIDS | cut -d' ' -f2)

# Run container on first MIG device
docker run --gpus "$MIG1" -it --name container1 nvidia/cuda:12.9.1-base-ubuntu24.04 nvidia-smi

# Run container on second MIG device
docker run --gpus "$MIG2" -it --name container2 nvidia/cuda:12.9.1-base-ubuntu24.04 nvidia-smi
```

## Considerations

- MIG partitions are fixed in size and cannot be resized after creation
- Each MIG instance has dedicated memory that cannot be oversubscribed
- Not all GPUs support MIG - check NVIDIA's documentation for supported hardware
- MIG mode must be enabled before creating partitions and cannot be disabled without rebooting
- On some GPUs (such as A30), MIG configurations persist until explicitly removed. On Blackwell generation, MIG configurations do not persist across reboots.

## References:

- [Use GPU MIG Partitioning (on SKS)](https://community.exoscale.com/product/compute/containers/how-to/gpu-mig-partitioning/).