Reference
Exoscale API
Schema Reference
Kafka Settings

Kafka Settings

Properties

PropertyTypeRequiredDescription
auto_create_topics_enablebooleannoEnable auto creation of topics
compression_typestringnoSpecify the final compression type for a given topic. This configuration accepts the standard compression codecs (‘gzip’, ‘snappy’, ’lz4’, ‘zstd’). It additionally accepts ‘uncompressed’ which is equivalent to no compression; and ‘producer’ which means retain the original compression codec set by the producer.

Allowed values: gzip, snappy, lz4, zstd, uncompressed, producer.
connections_max_idle_msintegernoIdle connections timeout: the server socket processor threads close the connections that idle for longer than this.
default_replication_factorintegernoReplication factor for autocreated topics
group_initial_rebalance_delay_msintegernoThe amount of time, in milliseconds, the group coordinator will wait for more consumers to join a new group before performing the first rebalance. A longer delay means potentially fewer rebalances, but increases the time until processing begins. The default value for this is 3 seconds. During development and testing it might be desirable to set this to 0 in order to not delay test execution time.
group_max_session_timeout_msintegernoThe maximum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures.
group_min_session_timeout_msintegernoThe minimum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures.
log-cleanup-and-compactionobjectnoschema details
log_flush_interval_messagesintegernoThe number of messages accumulated on a log partition before messages are flushed to disk
log_flush_interval_msintegernoThe maximum time in ms that a message in any topic is kept in memory before flushed to disk. If not set, the value in log.flush.scheduler.interval.ms is used
log_index_interval_bytesintegernoThe interval with which Kafka adds an entry to the offset index
log_index_size_max_bytesintegernoThe maximum size in bytes of the offset index
log_local_retention_bytesintegernoThe maximum size of local log segments that can grow for a partition before it gets eligible for deletion. If set to -2, the value of log.retention.bytes is used. The effective value should always be less than or equal to log.retention.bytes value.
log_local_retention_msintegernoThe number of milliseconds to keep the local log segments before it gets eligible for deletion. If set to -2, the value of log.retention.ms is used. The effective value should always be less than or equal to log.retention.ms value.
log_message_downconversion_enablebooleannoThis configuration controls whether down-conversion of message formats is enabled to satisfy consume requests.
log_message_timestamp_difference_max_msintegernoThe maximum difference allowed between the timestamp when a broker receives a message and the timestamp specified in the message
log_message_timestamp_typestringnoDefine whether the timestamp in the message is message create time or log append time.

Allowed values: CreateTime, LogAppendTime.
log_preallocatebooleannoShould pre allocate file when create new segment?
log_retention_bytesintegernoThe maximum size of the log before deleting messages
log_retention_hoursintegernoThe number of hours to keep a log file before deleting it
log_retention_msintegernoThe number of milliseconds to keep a log file before deleting it (in milliseconds), If not set, the value in log.retention.minutes is used. If set to -1, no time limit is applied.
log_roll_jitter_msintegernoThe maximum jitter to subtract from logRollTimeMillis (in milliseconds). If not set, the value in log.roll.jitter.hours is used
log_roll_msintegernoThe maximum time before a new log segment is rolled out (in milliseconds).
log_segment_bytesintegernoThe maximum size of a single log file
log_segment_delete_delay_msintegernoThe amount of time to wait before deleting a file from the filesystem
max_connections_per_ipintegernoThe maximum number of connections allowed from each ip address (defaults to 2147483647).
max_incremental_fetch_session_cache_slotsintegernoThe maximum number of incremental fetch sessions that the broker will maintain.
message_max_bytesintegernoThe maximum size of message that the server can receive.
min_insync_replicasintegernoWhen a producer sets acks to ‘all’ (or ‘-1’), min.insync.replicas specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful.
num_partitionsintegernoNumber of partitions for autocreated topics
offsets_retention_minutesintegernoLog retention window in minutes for offsets topic
producer_purgatory_purge_interval_requestsintegernoThe purge interval (in number of requests) of the producer request purgatory(defaults to 1000).
replica_fetch_max_bytesintegernoThe number of bytes of messages to attempt to fetch for each partition (defaults to 1048576). This is not an absolute maximum, if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that progress can be made.
replica_fetch_response_max_bytesintegernoMaximum bytes expected for the entire fetch response (defaults to 10485760). Records are fetched in batches, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that progress can be made. As such, this is not an absolute maximum.
sasl_oauthbearer_expected_audiencestringnoThe (optional) comma-delimited setting for the broker to use to verify that the JWT was issued for one of the expected audiences.
sasl_oauthbearer_expected_issuerstringnoOptional setting for the broker to use to verify that the JWT was created by the expected issuer.
sasl_oauthbearer_jwks_endpoint_urlstringnoOIDC JWKS endpoint URL. By setting this the SASL SSL OAuth2/OIDC authentication is enabled. See also other options for SASL OAuth2/OIDC.
sasl_oauthbearer_sub_claim_namestringnoName of the scope from which to extract the subject claim from the JWT. Defaults to sub.
socket_request_max_bytesintegernoThe maximum number of bytes in a socket request (defaults to 104857600).
transaction_partition_verification_enablebooleannoEnable verification that checks that the partition has been added to the transaction before writing transactional records to the partition
transaction_remove_expired_transaction_cleanup_interval_msintegernoThe interval at which to remove transactions that have expired due to transactional.id.expiration.ms passing (defaults to 3600000 (1 hour)).
transaction_state_log_segment_bytesintegernoThe transaction topic segment bytes should be kept relatively small in order to facilitate faster log compaction and cache loads (defaults to 104857600 (100 mebibytes)).
Last updated on