Managed OpenSearch default visualization

Each Managed OpenSearch plan comes with a default visualization with the included OpenSearch Dashboards.

The Dashboards interface is available as a second connection string from the service details. Use this URL to connect to the visualization tool and start making queries and reports on your data.

OpenSearch replication factor

The Managed OpenSearch plans have different replication factors:

  • Hobbyist and Startup: since those plans are single node based, no replication factor is supported.
  • Business and Premium: the replication factor can be adjusted to your application requirement from 1 replica to N-1 replicas where N is the total number of nodes in your cluster.

The Managed OpenSearch database service will automatically adjust the replication factor when the following conditions occur:

  • If number_of_replicas is too large for the current cluster size, it is automatically lowered to the maximum possible value (the number of nodes on the cluster - 1).
  • If number_of_replicas is 0 on a multi-node cluster, it is automatically increased to 1.
  • If number_of_replicas is between 1 and the maximum value, it is not adjusted.

When the replication factor (number_of_replicas value) is greater than the size of the cluster, number_of_replicas is automatically lowered, as it is not possible to replicate index shards to more nodes than there are on the cluster.

Specific options when creating or updating an OpenSearch service

You can find all the specific options for OpenSearch by using the CLI help as such:

exo dbaas create --help-opensearch
  --opensearch-dashboard-enabled                             Enable or disable OpenSearch Dashboards (default: true)                                                                                            
  --opensearch-dashboard-max-old-space-size                  Memory limit in MiB for OpenSearch Dashboards. Note: The memory reserved by OpenSearch Dashboards is not available for OpenSearch. (default: 128)  
  --opensearch-dashboard-request-timeout                     Timeout in milliseconds for requests made by OpenSearch Dashboards towards OpenSearch (default: 30000)                                             
  --opensearch-fork-from-service                             Service name                                                                                                                                       
  --opensearch-index-patterns                                JSON Array of index patterns (https://openapi-v2.exoscale.com/#operation-get-dbaas-service-opensearch-200-index-patterns)                          
  --opensearch-index-template-mapping-nested-objects-limit   The maximum number of nested cli-flag objects that a single document can contain across all nested types. Default is 10000.                        
  --opensearch-index-template-number-of-replicas             The number of replicas each primary shard has.                                                                                                     
  --opensearch-index-template-number-of-shards               The number of primary shards that an index should have.                                                                                            
  --opensearch-ip-filter                                     Allow incoming connections from CIDR address block                                                                                                 
  --opensearch-keep-index-refresh-interval                   index.refresh_interval is reset to default value for every index to be sure that indices are always visible to search. Set to true disable this.   
  --opensearch-max-index-count                               Maximum number of indexes to keep before deleting the oldest one                                                                                   
  --opensearch-recovery-backup-name                          Name of a backup to recover from for services that support backup names                                                                            
  --opensearch-settings                                      OpenSearch-specific settings (JSON)                                                                                                                
  --opensearch-version                                       OpenSearch major version      

Restricting connections from the Internet

By default, Exoscale DBaaS are not accessible from the whole Internet. Data does not transmit in clear over the network as it is SSL encrypted by default and authenticated.

To allow incoming connections to your database service, you can add a filter allowing:

  • just one IP address
  • a network range
  • a combination of the above

To do this update your service or create it with the ip filter which is a comma separated list of CIDRs exo dbaas update -z de-muc-1 test-opensearch --opensearch-ip-filter=1.2.3.4/24,5.6.7.8/32

OpenSearch vs Elasticsearch

OpenSearch is an open source fork original Elasticsearch project. When Elastic, the company behind the original Elasticsearch moved to a restrictive license, the changes meant that third parties were not able to offer Elasticsearch as a service. The community (including Exoscale DBaaS partner Aiven) joined forces to create and maintain OpenSearch based on the last open source licensed releases of both Elasticsearch and Kibana (v7.10.2) which are now OpenSearch and OpenSearch Dashboards.

Managed Opensearch additional configuration

You can list all settings available for the database service by using the command:

exo dbaas type show opensearch --settings opensearch
┼───────────────────────────────────────────────────────┼────────────────┼────────────────────────────────────────────────────┼
│                          KEY                          │      TYPE      │                    DESCRIPTION                     │
┼───────────────────────────────────────────────────────┼────────────────┼────────────────────────────────────────────────────┼
│ thread_pool_analyze_size                              │ integer        │ Size for the thread pool. See documentation for    │
│                                                       │                │ exact details. Do note this may have maximum value │
│                                                       │                │ depending on CPU count - value is automatically    │
│                                                       │                │ lowered if set to higher than maximum value.       │
│                                                       │                │   * Minimum: 1 / Maximum: 128                      │
│ search_max_buckets                                    │ [integer null] │ Maximum number of aggregation buckets allowed in a │
│                                                       │                │ single response. OpenSearch default value is used  │
│                                                       │                │ when this is not defined.                          │
│                                                       │                │   * Minimum: 1 / Maximum: 20000                    │
│                                                       │                │   * Example: 10000                                 │
│ email_sender_username                                 │ [string]       │                                                    │
│ thread_pool_force_merge_size                          │ integer        │ Size for the thread pool. See documentation for    │
│                                                       │                │ exact details. Do note this may have maximum value │
│                                                       │                │ depending on CPU count - value is automatically    │
│                                                       │                │ lowered if set to higher than maximum value.       │
│                                                       │                │   * Minimum: 1 / Maximum: 128                      │
│ email_sender_password                                 │ [string]       │ Sender email password for Opensearch alerts to     │
│                                                       │                │ authenticate with SMTP server                      │
│                                                       │                │   * Example: very-secure-mail-password             │
│ http_max_content_length                               │ integer        │ Maximum content length for HTTP requests to the    │
│                                                       │                │ OpenSearch HTTP API, in bytes.                     │
│                                                       │                │   * Minimum: 1 / Maximum: 2.147483647e+09          │
│ thread_pool_search_size                               │ integer        │ Size for the thread pool. See documentation for    │
│                                                       │                │ exact details. Do note this may have maximum value │
│                                                       │                │ depending on CPU count - value is automatically    │
│                                                       │                │ lowered if set to higher than maximum value.       │
│                                                       │                │   * Minimum: 1 / Maximum: 128                      │
│ indices_recovery_max_bytes_per_sec                    │ integer        │ Limits total inbound and outbound recovery traffic │
│                                                       │                │ for each node. Applies to both peer recoveries as  │
│                                                       │                │ well as snapshot recoveries (i.e., restores from a │
│                                                       │                │ snapshot). Defaults to 40mb                        │
│                                                       │                │   * Minimum: 40 / Maximum: 400                     │
│ email_sender_name                                     │ [string]       │ This should be identical to the Sender name        │
│                                                       │                │ defined in Opensearch dashboards                   │
│                                                       │                │   * Example: alert-sender                          │
│ thread_pool_analyze_queue_size                        │ integer        │ Size for the thread pool queue. See documentation  │
│                                                       │                │ for exact details.                                 │
│                                                       │                │   * Minimum: 10 / Maximum: 2000                    │
│ thread_pool_get_queue_size                            │ integer        │ Size for the thread pool queue. See documentation  │
│                                                       │                │ for exact details.                                 │
│                                                       │                │   * Minimum: 10 / Maximum: 2000                    │
│ indices_queries_cache_size                            │ integer        │ Percentage value. Default is 10%. Maximum amount   │
│                                                       │                │ of heap used for query cache. This is an expert    │
│                                                       │                │ setting. Too low value will decrease query         │
│                                                       │                │ performance and increase performance for other     │
│                                                       │                │ operations; too high value will cause issues with  │
│                                                       │                │ other OpenSearch functionality.                    │
│                                                       │                │   * Minimum: 3 / Maximum: 40                       │
│ http_max_initial_line_length                          │ integer        │ The max length of an HTTP URL, in bytes            │
│                                                       │                │   * Minimum: 1024 / Maximum: 65536                 │
│                                                       │                │   * Example: 4096                                  │
│ script_max_compilations_rate                          │ string         │ Script compilation circuit breaker limits the      │
│                                                       │                │ number of inline script compilations within a      │
│                                                       │                │ period of time. Default is use-context             │
│                                                       │                │   * Example: 75/5m                                 │
│ cluster_routing_allocation_node_concurrent_recoveries │ integer        │ How many concurrent incoming/outgoing shard        │
│                                                       │                │ recoveries (normally replicas) are allowed to      │
│                                                       │                │ happen on a node. Defaults to 2.                   │
│                                                       │                │   * Minimum: 2 / Maximum: 16                       │
│ thread_pool_search_queue_size                         │ integer        │ Size for the thread pool queue. See documentation  │
│                                                       │                │ for exact details.                                 │
│                                                       │                │   * Minimum: 10 / Maximum: 2000                    │
│ thread_pool_get_size                                  │ integer        │ Size for the thread pool. See documentation for    │
│                                                       │                │ exact details. Do note this may have maximum value │
│                                                       │                │ depending on CPU count - value is automatically    │
│                                                       │                │ lowered if set to higher than maximum value.       │
│                                                       │                │   * Minimum: 1 / Maximum: 128                      │
│ thread_pool_write_queue_size                          │ integer        │ Size for the thread pool queue. See documentation  │
│                                                       │                │ for exact details.                                 │
│                                                       │                │   * Minimum: 10 / Maximum: 2000                    │
│ action_auto_create_index_enabled                      │ boolean        │ Explicitly allow or block automatic creation of    │
│                                                       │                │ indices. Defaults to true                          │
│                                                       │                │   * Example: false                                 │
│ indices_query_bool_max_clause_count                   │ integer        │ Maximum number of clauses Lucene BooleanQuery can  │
│                                                       │                │ have. The default value (1024) is relatively high, │
│                                                       │                │ and increasing it may cause performance issues.    │
│                                                       │                │ Investigate other approaches first before          │
│                                                       │                │ increasing this value.                             │
│                                                       │                │   * Minimum: 64 / Maximum: 4096                    │
│ cluster_max_shards_per_node                           │ integer        │ Controls the number of shards allowed in the       │
│                                                       │                │ cluster per data node                              │
│                                                       │                │   * Minimum: 100 / Maximum: 10000                  │
│                                                       │                │   * Example: 1000                                  │
│ override_main_response_version                        │ boolean        │ Compatibility mode sets OpenSearch to report its   │
│                                                       │                │ version as 7.10 so clients continue to work.       │
│                                                       │                │ Default is false                                   │
│                                                       │                │   * Example: true                                  │
│ thread_pool_search_throttled_queue_size               │ integer        │ Size for the thread pool queue. See documentation  │
│                                                       │                │ for exact details.                                 │
│                                                       │                │   * Minimum: 10 / Maximum: 2000                    │
│ thread_pool_search_throttled_size                     │ integer        │ Size for the thread pool. See documentation for    │
│                                                       │                │ exact details. Do note this may have maximum value │
│                                                       │                │ depending on CPU count - value is automatically    │
│                                                       │                │ lowered if set to higher than maximum value.       │
│                                                       │                │   * Minimum: 1 / Maximum: 128                      │
│ reindex_remote_whitelist                              │ [array null]   │ Whitelisted addresses for reindexing. Changing     │
│                                                       │                │ this value will cause all OpenSearch instances to  │
│                                                       │                │ restart.                                           │
│ http_max_header_size                                  │ integer        │ The max size of allowed headers, in bytes          │
│                                                       │                │   * Minimum: 1024 / Maximum: 262144                │
│                                                       │                │   * Example: 8192                                  │
│ indices_recovery_max_concurrent_file_chunks           │ integer        │ Number of file chunks sent in parallel for each    │
│                                                       │                │ recovery. Defaults to 2.                           │
│                                                       │                │   * Minimum: 2 / Maximum: 5                        │
│ indices_fielddata_cache_size                          │ [integer null] │ Relative amount. Maximum amount of heap memory     │
│                                                       │                │ used for field data cache. This is an expert       │
│                                                       │                │ setting; decreasing the value too much will        │
│                                                       │                │ increase overhead of loading field data; too much  │
│                                                       │                │ memory used for field data cache will decrease     │
│                                                       │                │ amount of heap available for other operations.     │
│                                                       │                │   * Minimum: 3 / Maximum: 100                      │
│                                                       │                │   * Default: <nil>                                 │
│ action_destructive_requires_name                      │ [boolean null] │                                                    │
│ indices_memory_index_buffer_size                      │ integer        │ Percentage value. Default is 10%. Total amount of  │
│                                                       │                │ heap used for indexing buffer, before writing      │
│                                                       │                │ segments to disk. This is an expert setting. Too   │
│                                                       │                │ low value will slow down indexing; too high value  │
│                                                       │                │ will increase indexing performance but causes      │
│                                                       │                │ performance issues for query performance.          │
│                                                       │                │   * Minimum: 3 / Maximum: 40                       │
│ thread_pool_write_size                                │ integer        │ Size for the thread pool. See documentation for    │
│                                                       │                │ exact details. Do note this may have maximum value │
│                                                       │                │ depending on CPU count - value is automatically    │
│                                                       │                │ lowered if set to higher than maximum value.       │
│                                                       │                │   * Minimum: 1 / Maximum: 128                      │
┼───────────────────────────────────────────────────────┼────────────────┼────────────────────────────────────────────────────┼