Quick Start

In this quick start guide, you will create your first AI API Key and use it to target an Exoscale OpenAI-compatible endpoint.

Private Preview

On-Demand Inference is currently available as a private preview. Review the service boundaries before you begin.

This guide uses nvidia/Qwen3.8-27B-NVFP4 for a chat-completions request.

Prerequisites

You need:

the Exoscale CLI (exo) installed and configured
an Exoscale API key with access to the compute and ai services

First Steps via CLI

Configure API Access

On-demand Inference is managed through the Exoscale CLI and API.

Create a IAM role with compute and ai access:

exo iam role create ai-role --policy '{
  "default-service-strategy": "deny",
  "services": {
    "compute": {"type": "allow", "rules": []},
    "ai": {"type": "allow", "rules": []}
  }
}'

Create an IAM API key for the role:

exo iam api-key create ai-key ai-role

Add the key to your CLI configuration:

exo config add

Create an AI API Key

Create an AI API Key that grants access to all public models:

exo ai api-key create akey -z at-vie-2 --all-models

Without --all-models, --all-deployments, --model or --deployment, the key grants access to nothing. Alternatively, you can restrict the scope of an AI API key by specifying one or multiple public models:

exo ai api-key create akey -z at-vie-2 --model "nvidia/Qwen3.8-27B-NVFP4"
exo ai api-key create akey -z at-vie-2 \
  --model "nvidia/Qwen3.8-27B-NVFP4" \
  --model "Qwen/Qwen3-Embedding-8B"

┼─────────────────┼──────────────────────────────────────|
│ ID              │ 9f5d4a1e-31aa-413e-9136-d9d36eabffe1 │
│ Name            │ akey                                 │
| Models          | nvidia/Qwen3.8-27B-NVFP4             |
|                 | Qwen/Qwen3-Embedding-8B              |
| Deployments     | n/a                                  |
| All Models      | false                                |
| All Deployments | false                                |
┼─────────────────┼──────────────────────────────────────|

Note: The scope of an AI API key can be updated later on via the update command.

Use Your AI API Key on an On-Demand Endpoint

Export both values:

export ENDPOINT_URL="https://inference.at-vie-2.exoscale-cloud.com/v1"
export API_KEY="<ai-api-key>"

Always treat the AI API key as a secret.

Send a Test Request

On-demand Inference endpoints are OpenAI-compatible. You can call the chat completions route with a standard OpenAI-style request body.

curl -X POST "$ENDPOINT_URL/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $API_KEY" \
  -d '{
    "model": "nvidia/Qwen3.8-27B-NVFP4",
    "messages": [
      {
        "role": "user",
        "content": "Write a short poem about Switzerland."
      }
    ],
    "max_tokens": 300
  }'

The hosted Qwen3.8-27B model supports text generation and reasoning, image and video understanding, coding, tool calling, structured output, and streaming.

Supported Chat Completions Parameters

The private-preview /v1/chat/completions endpoint supports the following OpenAI-compatible request parameters for Qwen3.8-27B:

Purpose	Parameters
Request	`model`, `messages`, `user`
Output length and sampling	`max_tokens`, `temperature`, `top_p`, `n`, `stop`, `seed`
Penalties	`frequency_penalty`, `presence_penalty`
Streaming	`stream`, `stream_options`
Reasoning	`reasoning_effort` (`low`, `medium`, or `xhigh`)
Tools	`tools`, `tool_choice`, `parallel_tool_calls`
Output format	`response_format`
Token probabilities	`logprobs`, `top_logprobs`
Qwen extensions	`top_k`, `min_p`, `repetition_penalty`, `chat_template_kwargs.enable_thinking`, `chat_template_kwargs.preserve_thinking`

Text, image, or video content is supplied through messages. Use exo ai model show to inspect the capabilities advertised by the currently hosted version, and review Limits and Quotas for the current service boundaries.

Send an Embedding Request

Use the embeddings route with Qwen/Qwen3-Embedding-8B:

curl -X POST "$ENDPOINT_URL/embeddings" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $API_KEY" \
  -d '{
    "model": "Qwen/Qwen3-Embedding-8B",
    "input": "Vienna is the capital of Austria.",
    "dimensions": 1024,
    "encoding_format": "float"
  }'

The model accepts up to 32,768 tokens and supports output dimensions from 32 through 4096. The embeddings endpoint accepts model, input, dimensions, encoding_format, and user.

Cache reads are applied automatically where supported and do not require a request parameter. A cache read occurs when a request reuses eligible prompt tokens already stored in the cache. These tokens are charged at the lower cache-read price instead of the standard input-token price. Cache-write support, including TTL configuration, is coming soon.

First Steps via Portal

On-Demand Inference integration with the Exoscale Portal is coming soon. Until then, use the Exoscale CLI or API to manage AI API keys and access public models.

Last updated on October 22, 2026