---
title: "Quick Start"
description: "Create an AI API Key and send your first inference request to the Exoscale On-demand Inference platform."
url: https://community.exoscale.com/product/concrete-ai/on-demand-inference/quick-start/
product: "On-Demand Inference"
section: product
last_updated: 2026-10-22
---
> For AI agents: the documentation index is at https://community.exoscale.com/llms.txt. Every page is available as markdown at `<page URL>index.md` or with `Accept: text/markdown`.

# Quick Start

In this quick start guide, you will create your first AI API Key and use it to
target an Exoscale OpenAI-compatible endpoint.

> [!IMPORTANT]
> **Private Preview**
>
> On-Demand Inference is currently available as a private preview. Review the
> [service boundaries](/product/concrete-ai/on-demand-inference/service-boundaries/index.md) before you
> begin.

This guide uses `nvidia/Qwen3.8-27B-NVFP4` for a chat-completions request.

## Prerequisites

You need:

* the Exoscale CLI (`exo`) [installed and configured](/reference/cli/exo/index.md)
* an Exoscale API key with access to the `compute` and `ai` services

## First Steps via CLI

### Configure API Access

On-demand Inference is managed through the Exoscale CLI and API.

Create a IAM role with `compute` and `ai` access:

```bash
exo iam role create ai-role --policy '{
  "default-service-strategy": "deny",
  "services": {
    "compute": {"type": "allow", "rules": []},
    "ai": {"type": "allow", "rules": []}
  }
}'
```

Create an IAM API key for the role:

```bash
exo iam api-key create ai-key ai-role
```

Add the key to your CLI configuration:

```bash
exo config add
```


### Create an AI API Key

Create an AI API Key that grants access to all public models:

```bash
exo ai api-key create akey -z at-vie-2 --all-models
```

Without `--all-models`, `--all-deployments`, `--model` or `--deployment`, the key
grants access to nothing. Alternatively, you can restrict the scope of an AI API
key by specifying one or multiple public models:


```bash
exo ai api-key create akey -z at-vie-2 --model "nvidia/Qwen3.8-27B-NVFP4"
exo ai api-key create akey -z at-vie-2 \
  --model "nvidia/Qwen3.8-27B-NVFP4" \
  --model "Qwen/Qwen3-Embedding-8B"
```
```shell
┼─────────────────┼──────────────────────────────────────|
│ ID              │ 9f5d4a1e-31aa-413e-9136-d9d36eabffe1 │
│ Name            │ akey                                 │
| Models          | nvidia/Qwen3.8-27B-NVFP4             |
|                 | Qwen/Qwen3-Embedding-8B              |
| Deployments     | n/a                                  |
| All Models      | false                                |
| All Deployments | false                                |
┼─────────────────┼──────────────────────────────────────|
```

Note: The scope of an AI API key can be updated later on via the `update` command.


### Use Your AI API Key on an On-Demand Endpoint

Export both values:

```bash
export ENDPOINT_URL="https://inference.at-vie-2.exoscale-cloud.com/v1"
export API_KEY="<ai-api-key>"
```

Always treat the AI API key as a secret.

### Send a Test Request

On-demand Inference endpoints are OpenAI-compatible. You can call the chat completions route with a standard OpenAI-style request body.

```bash
curl -X POST "$ENDPOINT_URL/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $API_KEY" \
  -d '{
    "model": "nvidia/Qwen3.8-27B-NVFP4",
    "messages": [
      {
        "role": "user",
        "content": "Write a short poem about Switzerland."
      }
    ],
    "max_tokens": 300
  }'
```

The hosted Qwen3.8-27B model supports text generation and reasoning, image and
video understanding, coding, tool calling, structured output, and streaming.

#### Supported Chat Completions Parameters

The private-preview `/v1/chat/completions` endpoint supports the following
OpenAI-compatible request parameters for Qwen3.8-27B:

| Purpose | Parameters |
| :--- | :--- |
| Request | `model`, `messages`, `user` |
| Output length and sampling | `max_tokens`, `temperature`, `top_p`, `n`, `stop`, `seed` |
| Penalties | `frequency_penalty`, `presence_penalty` |
| Streaming | `stream`, `stream_options` |
| Reasoning | `reasoning_effort` (`low`, `medium`, or `xhigh`) |
| Tools | `tools`, `tool_choice`, `parallel_tool_calls` |
| Output format | `response_format` |
| Token probabilities | `logprobs`, `top_logprobs` |
| Qwen extensions | `top_k`, `min_p`, `repetition_penalty`, `chat_template_kwargs.enable_thinking`, `chat_template_kwargs.preserve_thinking` |

Text, image, or video content is supplied through `messages`. Use
`exo ai model show` to inspect the capabilities advertised by the currently
hosted version, and review [Limits and Quotas](/product/concrete-ai/on-demand-inference/service-boundaries/limits-and-quotas/index.md) for the current service boundaries.

### Send an Embedding Request

Use the embeddings route with `Qwen/Qwen3-Embedding-8B`:

```bash
curl -X POST "$ENDPOINT_URL/embeddings" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $API_KEY" \
  -d '{
    "model": "Qwen/Qwen3-Embedding-8B",
    "input": "Vienna is the capital of Austria.",
    "dimensions": 1024,
    "encoding_format": "float"
  }'
```

The model accepts up to 32,768 tokens and supports output dimensions from 32
through 4096. The embeddings endpoint accepts `model`, `input`, `dimensions`,
`encoding_format`, and `user`.

Cache reads are applied automatically where supported and do not require a
request parameter. A cache read occurs when a request reuses eligible prompt
tokens already stored in the cache. These tokens are charged at the lower
[cache-read price](https://community.exoscale.com/product/concrete-ai/on-demand-inference/service-boundaries/limits-and-quotas/index.md/index.md#tentative-pricing) instead of the standard input-token price. Cache-write support, including
TTL configuration, is coming soon.

## First Steps via Portal

On-Demand Inference integration with the Exoscale Portal is coming soon. Until
then, use the Exoscale CLI or API to manage AI API keys and access public
models.