# Managed ClickHouse® metrics available via Prometheus

Managed ClickHouse® services can expose their metrics on a Prometheus scrape endpoint. The
endpoint is not enabled by default: you create a Prometheus external endpoint, attach your
service to it, then scrape it.

## Enable the Prometheus endpoint

1.  Create a Prometheus external endpoint with the credentials your Prometheus server uses:

    ```bash
    exo dbaas external-endpoint create prometheus my-prometheus \
      --prometheus-basic-auth-username PROMETHEUS_USER \
      --prometheus-basic-auth-password PROMETHEUS_PASSWORD
    ```

1.  Attach your Managed ClickHouse service to that endpoint, using the endpoint ID returned
    above (`exo dbaas external-endpoint list` shows it again):

    ```bash
    exo dbaas external-integration attach prometheus \
      --source-service-name my-clickhouse \
      --destination-endpoint-id ENDPOINT_ID
    ```

1.  Read the scrape host and port from the service. They are published as `prometheus-uri`,
    and the port is not part of the service `components` list:

    ```bash
    exo x get-dbaas-service-clickhouse my-clickhouse -z ch-gva-2 -q '"prometheus-uri"'
    ```

## Scrape the endpoint

The endpoint uses TLS with a certificate issued by the certificate authority of your
organization's DBaaS services, and it
is protected by the basic authentication credentials of the external endpoint. Retrieve the
CA certificate first:

```bash
exo dbaas ca-certificate -z ch-gva-2 > ca.pem
```

Use the host and port published in `prometheus-uri` as `CLICKHOUSE_HOSTNAME` and
`PROMETHEUS_PORT`. The port was `9273` on the services used for this guide.

Then request the metrics:

```bash
curl --cacert ca.pem \
    --user 'PROMETHEUS_USER:PROMETHEUS_PASSWORD' \
    'https://CLICKHOUSE_HOSTNAME:PROMETHEUS_PORT/metrics'
```

> [!IMPORTANT]
> The scrape endpoint is governed by the service
> [IP filter](/product/dbaas/service-specific/clickhouse/how-to/secure-service/): add the public
> IP address of your Prometheus server to it. A scrape that hangs and then times out with no TLS
> or authentication error means the scraping host is not allowed by the IP filter.

The response is standard Prometheus exposition format, with one label set per service and node:

```text
# HELP clickhouse_metrics_query Telegraf collected metric
# TYPE clickhouse_metrics_query untyped
clickhouse_metrics_query{cloud="exoscale-ch-gva-2",host="my-clickhouse-1",project="exoscale-ORGANIZATION_ID",service="my-clickhouse",service_type="clickhouse"} 6
```

## Exposed metrics

The exposed metric set depends on the plan and the topology of your service: single-node
services expose no Keeper or replication queue metrics, while multi-node services add them.
The metrics come from the following families:

| Family | Content |
|---|---|
| `clickhouse_*` | ClickHouse events, metrics and asynchronous metrics |
| `zookeeper_*` | Keeper ensemble metrics (multi-node topologies) |
| `diskio_*`, `mem_*`, `net_*`, `netstat_*`, `processes_*`, `system_*` | Host level metrics collected by the agent |

Retrieve the exact list for your own service with the `curl` command above, for example:

```bash
curl -s --cacert ca.pem --user 'PROMETHEUS_USER:PROMETHEUS_PASSWORD' \
    'https://CLICKHOUSE_HOSTNAME:PROMETHEUS_PORT/metrics' | grep '^# HELP' | awk '{print $3}' | sort -u
```

## Related pages

- [Managed ClickHouse® metrics available in Datadog](/product/dbaas/service-specific/clickhouse/reference/clickhouse-metrics-datadog/)
- [Use External Integrations](/product/dbaas/how-to/external-integrations/)