# Configure OCI registries pull-through cache

Pulling images directly from public registries works, but it is rarely the best fit in production. Routing container image pulls through a private mirror helps you reduce egress costs, control which images are allowed in your cluster, and avoid hitting rate limits imposed by upstream registries. With Karpenter on Exoscale SKS, mirror endpoints and pull credentials are configured through the `ExoscaleNodeClass` resource.

## Prerequisites

As a prerequisite for the following documentation, you need:

- An Exoscale SKS cluster on the Pro plan with the Karpenter addon enabled.
- Access to your cluster with `kubectl`.
- A Karpenter `NodePool` and an `ExoscaleNodeClass` configured for Karpenter.
- A reachable OCI-compliant registry to use as a mirror.
- Basic Kubernetes and YAML knowledge.

If you do not have access to an SKS cluster, follow the [Quick Start Guide]({{< ref "/product/compute/instances/quick-start/" >}}).

## How registry mirroring works

The `ExoscaleNodeClass` exposes a structured `spec.containerRegistry` field. Karpenter renders it into the SKS node agent configuration, which configures container runtime registry mirrors. When it resolves an image reference, it first queries the configured mirror endpoints. If the mirror returns a manifest, it pulls the image from there. Otherwise, the request falls back to the upstream registry.

Two kinds of configuration are supported:

- `mirrors`: per-registry mirror endpoints, with optional TLS material to authenticate the mirror.
- `credentials`: per-registry pull credentials (basic auth, registry auth token, or identity token) used when contacting a private registry.

Both `mirrors` and `credentials` reference Kubernetes Secrets that must live in the `kube-system` namespace. Karpenter validates the secrets and the referenced keys at reconciliation time. If a secret is missing or malformed, the `ExoscaleNodeClass` reports a `ContainerRegistrySecretsResolved=False` condition with a precise reason (for example `SecretMissing`, `InvalidPEM`, `TLSSecretInconsistent`).

> [!IMPORTANT]
> `spec.containerRegistry` takes priority over `spec.userData`, always configure mirroring in a single location, preferably `spec.containerRegistry`.

## Configure a mirror

The following example routes pulls for `docker.io` and `ghcr.io` to a private mirror served over HTTPS at `https://mirror.example.com`. The mirror exposes a public CA bundled by the cluster, so no TLS secret is required.

```yaml
apiVersion: karpenter.exoscale.com/v1
kind: ExoscaleNodeClass
metadata:
  name: with-mirror
spec:
  imageTemplateSelector: {}

  containerRegistry:
    mirrors:
      - registry: docker.io
        endpoints:
          - url: https://mirror.example.com
      - registry: ghcr.io
        endpoints:
          - url: https://mirror.example.com
```

Each `mirrors` entry accepts one to ten `endpoints`. When several endpoints are listed, the container runtimes tries them in order and falls back to the next one on failure. This is useful to expose a primary mirror and a backup mirror in a different region or zone.

### Use a private mirror with a custom CA

When the mirror serves a certificate signed by a private CA, the CA bundle must be mounted on every Karpenter node. Create a Secret in `kube-system` containing the CA, then reference it from the mirror endpoint:

```bash
kubectl -n kube-system create secret generic mirror-ca \
  --from-file=ca.crt=./mirror-ca.pem
```

```yaml
apiVersion: karpenter.exoscale.com/v1
kind: ExoscaleNodeClass
metadata:
  name: with-mirror
spec:
  imageTemplateSelector: {}

  containerRegistry:
    mirrors:
      - registry: docker.io
        endpoints:
          - url: https://mirror.example.com
            tlsSecretRef:
              name: mirror-ca
```

The referenced Secret must live in `kube-system` and contain at least `ca.crt` as a PEM-encoded certificate. Mutual TLS is supported by adding `tls.crt` and `tls.key` to the same Secret: both keys must be set together, otherwise reconciliation fails with `TLSSecretInconsistent`.

When the mirror preserves the upstream path (`https://mirror.example.com/v2/...`), you can leave `overridePath` unset. If the mirror exposes a flattened path layout (for example mapping multiple registries under a single namespace), set `overridePath: true` on the endpoint.

> [!WARNING]
> Avoid `skipVerify: true` in production. It disables certificate validation against the mirror and should only be used for ephemeral or test setups.

## Configure pull credentials

To pull from a private registry, configure a credential entry. Three authentication schemes are supported, and exactly one of `basic`, `auth`, or `identityToken` must be set per credential:

```bash
kubectl -n kube-system create secret generic registry-pull \
  --from-literal=username='robot' \
  --from-literal=password='secret-token'
```

```yaml
apiVersion: karpenter.exoscale.com/v1
kind: ExoscaleNodeClass
metadata:
  name: with-mirror
spec:
  imageTemplateSelector: {}

  containerRegistry:
    credentials:
      - registry: reg.example.com
        basic:
          usernameSecretRef:
            name: registry-pull
            key: username
          passwordSecretRef:
            name: registry-pull
            key: password
```

The `auth` scheme uses a single key (typically the pre-encoded `user:password` string used by the Docker registry `WWW-Authenticate` flow):

```yaml
credentials:
  - registry: reg.example.com
    auth:
      authSecretRef:
        name: registry-pull
        key: auth
```

The `identityToken` scheme is intended for short-lived OIDC-based tokens:

```yaml
credentials:
  - registry: my-oidc-registry.example.com
    identityToken:
      identityTokenSecretRef:
        name: ecr-identity-token
        key: identitytoken
```

Referenced Secrets must exist in `kube-system` and contain the requested key. The reconciler returns a `SecretMissing` or `SecretKeyMissing` reason when the lookup fails.

## Deploy the configuration

Apply the `ExoscaleNodeClass` before applying or updating a `NodePool` that references it:

```bash
kubectl apply -f exoscale-nodeclass.yaml
kubectl apply -f nodepool.yaml
```

A `NodePool` must reference the node class through `nodeClassRef`:

```yaml
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: mirrored
spec:
  template:
    spec:
      nodeClassRef:
        group: karpenter.exoscale.com
        kind: ExoscaleNodeClass
        name: with-mirror
      requirements:
        - key: node.kubernetes.io/instance-type
          operator: In
          values:
            - standard.medium
            - standard.large
```

New nodes provisioned for this `NodePool` start with the configured mirrors and credentials. Existing nodes are not reconfigured in place. When the mirror configuration or any referenced Secret changes, Karpenter detects the drift through the `ContainerRegistrySecretsResolved` condition and replaces affected nodes when disruption policies allow it.

## Verify the configuration

Check that the node class reports ready:

```bash
kubectl get exoscalenodeclass with-mirror \
  -o jsonpath='{.status.conditions[?(@.type=="ContainerRegistrySecretsResolved")]}'
```

> [!WARNING]
> Changing the registry configuration affects node allocation and will cause Karpenter to replace nodes. Plan the change around your workload disruption budgets and verify that enough capacity remains available during the replacement.