9 Aug 2022

Aiven finishes the transition away from Elasticsearch: technical details

On 2022-08-23, Aiven will complete its migration away from Elasticsearch. Read on to find out how this affects the backwards compatibility we were providing.

hoang-minh-vo

Hoang Minh Vo

|

RSS Feed

Technical Product Manager

Aiven finishes the transition away from Elasticsearch: technical details illustration

In September 2021, Aiven introduced the Aiven for OpenSearch® service as a successor to Aiven for Elasticsearch. In order to ease the migration, Aiven provides an upgrade path from Aiven for Elasticsearch to Aiven for OpenSearch that maintains backward capability to Aiven for Elasticsearch API and metrics.

On 23 August 2022, in order to complete the migration, Aiven sunsets the backward compatible capability to all migrated Aiven for OpenSearch clusters. This article explains what changes are necessary after performing the update.

After 23 August 2022, Aiven for Elasticsearch API is no longer supported. We recommend that you switch to the Aiven for OpenSearch® API instead as soon as possible. This includes using GET, PUT and POST to:

  • https://api.aiven.io/v1/project/{project}/service/{service_name}/opensearch/acl

instead of:

  • https://api.aiven.io/v1/project/{project}/service/{service_name}/elasticsearch/acl

You can check the usage of the Aiven API for Aiven for OpenSearch in the API reference.

Sunsetting the backwards compatibility with Aiven for Elasticsearch changes how to access service information using the Aiven API. When you make a request to https://api.aiven.io/v1/project/{project}/service/{service_name}, you can see the following changes:

  • elasticsearch and kibana are no longer returned under the components field
  • elasticsearch, elasticsearch_username, elasticsearch_password and kibana_uri are no longer returned under connection_info
  • elasticsearch_version is no longer returned under user_config
  • service_type returns opensearch (instead of elasticsearch)

An example of a JSON response to a backward compatible with Aiven for Elasticsearch Aiven for OpenSearch® service:

{ "service_type": "elasticsearch", "components": [ { "component": "elasticsearch", "host": "ELASTICSEARCH_HOST", "port": 12691, "privatelink_connection_id": null, "route": "dynamic", "usage": "primary" } { "component": "kibana", "host": "ELASTICSEARCH_HOST", "port": 443, "privatelink_connection_id": null, "route": "dynamic", "usage": "primary" }, { "component": "opensearch", "host": "OPENSEARCH_HOST", "port": 12691, "privatelink_connection_id": null, "route": "dynamic", "usage": "primary" } { "component": "opensearch_dashboards", "host": "OPENSEARCH_HOST", "port": 443, "privatelink_connection_id": null, "route": "dynamic", "usage": "primary" } ] "connection_info":{ "elasticsearch": [], "kibana_uri": "ELASTICSEARCH_URI", "elasticsearch_password":"PASSWORD", "elasticsearch_username":"USERNAME", "opensearch": [], "opensearch_dashboards_uri": "OPENSEARCH_URI", "opensearch_password":"PASSWORD", "opensearch_username":"USERNAME" }, "user_config": { "elasticsearch_version": 1.3.3 } }

An example of a JSON response to an Aiven for OpenSearch® service after backward compatibility with Elasticsearch has been turned off:

{ "service_type": "opensearch", "components": [ { "component": "opensearch", "host": "OPENSEARCH_HOST", "port": 12691, "privatelink_connection_id": null, "route": "dynamic", "usage": "primary" } { "component": "opensearch_dashboards", "host": "OPENSEARCH_HOST", "port": 443, "privatelink_connection_id": null, "route": "dynamic", "usage": "primary" } ] "connection_info":{ "opensearch": [], "opensearch_dashboards_uri": "OPENSEARCH_URI", "opensearch_password":"PASSWORD", "opensearch_username":"USERNAME" }, "user_config": { "opensearch_version": 1.3.3 } }

Once the backward compatibility is turned off, you can see the differences on your OpenSearch® service page in Aiven Console: Elasticsearch version becomes OpenSearch version.

  • Service page for Backward compatible Aiven for OpenSearch®:

    A screenshot of the OpenSearch® Service page for Backward Compatible OpenSearch®

  • Service page for Aiven for OpenSearch® after turning off backward compatibility:

    A screenshot of the OpenSearch® Service page for Pure OpenSearch®

Aiven provides metrics via the Telegraf plugin, so metrics that are available across Aiven for InfluxDB®, Aiven for M3 metrics integration, external Prometheus integration, and external AWS CloudWatch metrics integration remain the same. You can see the full list of detail metrics and additional metrics.

Once we turn off backward compatibility, Aiven for OpenSearch® will not produce any metrics prefixed with elasticsearch_ to any types of metric integrations apart from an external Datadog integration. Therefore, the metrics from your Aiven for OpenSearch® cluster will only contain the opensearch_ prefixes from the above lists.

Aiven does not have control over Datadog agents, therefore all metrics sent to external Datadog metrics integration will still still have elasticsearch. prefixes.

If you have a default Aiven for Grafana® dashboard, Aiven automatically converts all queries using elasticsearch_ prefixes to opensearch_ prefixes.

If you have a non-default Aiven for Grafana® dashboard, Aiven provides a tool to help convert your Grafana® dashboard from using elasticsearch_ to opensearch_. See the instructions for replacing the expression string for more information.

To find out how to migrate external Elasticsearch data, follow the instructions for migrating Elasticsearch data to Aiven for OpenSearch®.


Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries.

Related blogs

All things open source, plus our product updates and news in a monthly newsletter.

Subscribe to the Aiven newsletter

Loading...