Aiven finishes the transition away from Elasticsearch: technical details

Technical details about completing Aiven's migration away from Elasticsearch

In September 2021, Aiven introduced the Aiven for OpenSearch® service as a successor to Aiven for Elasticsearch. In order to ease the migration, Aiven provides an upgrade path from Aiven for Elasticsearch to Aiven for OpenSearch that maintains backward capability to Aiven for Elasticsearch API and metrics.

On 23 August 2022, in order to complete the migration, Aiven sunsets the backward compatible capability to all migrated Aiven for OpenSearch clusters. This article explains what changes are necessary after performing the update.

Aiven REST API

Aiven for OpenSearch^® ACL API

After 23 August 2022, Aiven for Elasticsearch API is no longer supported. We recommend that you switch to the Aiven for OpenSearch® API instead as soon as possible. This includes using GET, PUT and POST to:

• https://api.aiven.io/v1/project/{project}/service/{service_name}/opensearch/acl

instead of:

• https://api.aiven.io/v1/project/{project}/service/{service_name}/elasticsearch/acl

You can check the usage of the Aiven API for Aiven for OpenSearch in the API reference.

Aiven API to access service information

Sunsetting the backwards compatibility with Aiven for Elasticsearch changes how to access service information using the Aiven API. When you make a request to https://api.aiven.io/v1/project/{project}/service/{service_name}, you can see the following changes:

• elasticsearch and kibana are no longer returned under the components field
• elasticsearch, elasticsearch_username, elasticsearch_password and kibana_uri are no longer returned under connection_info
• elasticsearch_version is no longer returned under user_config
• service_type returns opensearch (instead of elasticsearch)

An example of a JSON response to a backward compatible with Aiven for Elasticsearch Aiven for OpenSearch® service:

    {
     "service_type": "elasticsearch",
     "components": [
      {
       "component": "elasticsearch",
           "host": "ELASTICSEARCH_HOST",
           "port": 12691,
           "privatelink_connection_id": null,
           "route": "dynamic",
           "usage": "primary"
      }
      {
           "component": "kibana",
           "host": "ELASTICSEARCH_HOST",
           "port": 443,
           "privatelink_connection_id": null,
           "route": "dynamic",
           "usage": "primary"
      },
      {
       "component": "opensearch",
           "host": "OPENSEARCH_HOST",
           "port": 12691,
           "privatelink_connection_id": null,
           "route": "dynamic",
           "usage": "primary"
      }
      {
           "component": "opensearch_dashboards",
           "host": "OPENSEARCH_HOST",
           "port": 443,
           "privatelink_connection_id": null,
           "route": "dynamic",
           "usage": "primary"
      }
     ]
     "connection_info":{
      "elasticsearch": [],
      "kibana_uri": "ELASTICSEARCH_URI",
      "elasticsearch_password":"PASSWORD",
      "elasticsearch_username":"USERNAME",
      "opensearch": [],
      "opensearch_dashboards_uri": "OPENSEARCH_URI",
      "opensearch_password":"PASSWORD",
      "opensearch_username":"USERNAME"
     },
     "user_config": {
      "elasticsearch_version": 1.3.3
     }
    }
Copy to clipboard

An example of a JSON response to an Aiven for OpenSearch® service after backward compatibility with Elasticsearch has been turned off:

    {
     "service_type": "opensearch",
     "components": [
      {
       "component": "opensearch",
           "host": "OPENSEARCH_HOST",
       "port": 12691,
           "privatelink_connection_id": null,
           "route": "dynamic",
           "usage": "primary"
      }
      {
           "component": "opensearch_dashboards",
           "host": "OPENSEARCH_HOST",
           "port": 443,
           "privatelink_connection_id": null,
           "route": "dynamic",
           "usage": "primary"
      }
     ]
     "connection_info":{
      "opensearch": [],
      "opensearch_dashboards_uri": "OPENSEARCH_URI",
      "opensearch_password":"PASSWORD",
      "opensearch_username":"USERNAME"
     },
     "user_config": {
      "opensearch_version": 1.3.3
     }
    }
Copy to clipboard

Aiven Console

Once the backward compatibility is turned off, you can see the differences on your OpenSearch® service page in Aiven Console: Elasticsearch version becomes OpenSearch version.

• Service page for Backward compatible Aiven for OpenSearch®:

  

• Service page for Aiven for OpenSearch® after turning off backward compatibility:

  

Metrics integrations

Aiven provides metrics via the Telegraf plugin, so metrics that are available across Aiven for InfluxDB®, Aiven for M3 metrics integration, external Prometheus integration, and external AWS CloudWatch metrics integration remain the same. You can see the full list of
detail metrics and
additional metrics.

Once we turn off backward compatibility, Aiven for OpenSearch® will not produce any metrics prefixed with elasticsearch_ to any types of metric integrations apart from an external Datadog integration. Therefore, the metrics from your Aiven for OpenSearch® cluster will only contain the opensearch_ prefixes from the above lists.

Datadog metrics integrations

Aiven does not have control over Datadog agents, therefore all metrics sent to external Datadog metrics integration will still still have elasticsearch. prefixes.

Grafana^®

If you have a default Aiven for Grafana® dashboard, Aiven automatically converts all queries using elasticsearch_ prefixes to opensearch_ prefixes.

If you have a non-default Aiven for Grafana® dashboard, Aiven provides a
tool to help convert your Grafana® dashboard from using elasticsearch_ to opensearch_. See the
instructions for replacing the expression string for more information.

Further reading

To find out how to migrate external Elasticsearch data, follow the instructions for migrating Elasticsearch data to Aiven for OpenSearch®.

Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries.