Get started with Aiven for OpenSearch®

Learn how to use Aiven for OpenSearch®, create a service, secure access, manage indices, and explore your data.

Aiven for OpenSearch® is a fully managed OpenSearch service designed for reliability, scalability, and security. It includes OpenSearch Dashboards for data visualization and supports integrations for logs and monitoring.

Prerequisites

Ensure you have the following before getting started:

• Access to the Aiven Console
• Aiven CLI installed
• Aiven Provider for Terraform installed
• API token

Create an Aiven for OpenSearch® service

Console

1. From your project, in the Services page, click Create service.

2. From the Select service page, click OpenSearch®.

3. Select the cloud provider and region to host your service on.

   note

   The pricing for the same service can vary between different providers and regions. The service summary shows you the pricing for your selected options.

4. Select a service plan.

   note

   This determines the number of servers and the memory, CPU, and disk resources allocated to your service. See Plans & Pricing.

5. Optional: Add disk storage.

6. Enter a name for your service.

   important

   You cannot change the name after you create the service.

   You can fork the service with a new name instead.

7. Optional: Add tags.

8. Click Create service.

The Overview page of the service opens. It shows the connection parameters for your service, its current status, and the configuration options.

The status of the service is Rebuilding during its creation. When the status becomes Running, you can start using the service. This typically takes couple of minutes and can vary between cloud providers and regions.

API

Create the service using the Aiven API, run:

curl -X POST https://api.aiven.io/v1/project/<project-name>/service \
     -H "Authorization: Bearer <api-token>" \
     -H "Content-Type: application/json" \
     -d '{
           "cloud": "google-europe-west1",
           "plan": "startup-4",
           "service_name": "example-opensearch",
           "service_type": "opensearch"
         }'

Parameters:

• <project-name>: Your project name.
• <api-token>: Your API token.

CLI

Create the service using the Aiven CLI, run:

avn service create <service-name> \
  --service-type opensearch       \
  --cloud <cloud-region>          \
  --plan <service-plan>

Parameters:

• <service-name>: Name of your service (for example, my-opensearch).
• <cloud-region>: Deployment region (for example, google-europe-west1).
• <service-plan>: Subscription plan (for example, startup-4).

Terraform

1. Create a main.tf file with the following content:

    variable "aiven_token" {
      type = string
    }
   
    variable "aiven_project_name" {
      type = string
    }
   
    terraform {
      required_providers {
        aiven = {
          source  = "aiven/aiven"
          version = ">=4.0.0"
        }
      }
    }
   
    provider "aiven" {
      api_token = var.aiven_token
    }
   
    resource "aiven_opensearch" "example" {
      project      = var.aiven_project_name
      service_name = "example-opensearch"
      cloud_name   = "google-europe-west1"
      plan         = "startup-4"
    }

2. Create a terraform.tfvars file:

    aiven_token = "your-api-token"
    aiven_project_name = "your-project-name"

3. Run the Terraform commands to apply the configuration:

   terraform init
   terraform apply --auto-approve

Secure access with ACLs

Secure your service by using one of the following options:

• Access control lists (ACLs): Manage access to indices by setting patterns (for example, logs-*) and permissions (read, write, or all) in the Aiven Console.
• OpenSearch security: Use OpenSearch Dashboards or APIs for fine-grained access control, including role-based access control (RBAC) and single sign-on (SSO).

For more information, see Access control in Aiven for OpenSearch®.

Manage indices

Aiven for OpenSearch® lets you view and manage indices and configure index retention patterns. For detailed steps on creating and managing indices, see the OpenSearch documentation.

View and manage indices

1. Open your service in the Aiven Console.
2. Click Indexes to view details such as shards, replicas, size, and health.

Configure retention patterns

1. In Indexes page, scroll to Index retention patterns
2. Click Add pattern and define:
   • Pattern: Specify index patterns (for example, *_logs_*).
   • Maximum index count: Set the number of indices to retain.
3. Click Create to save.

For advanced indexing features, including custom mappings, refer to the OpenSearch documentation.

Access OpenSearch Dashboards

Use OpenSearch Dashboards to visualize and analyze your data.

1. On the Overview page service in the Aiven Console.
2. In the Connection information section, click the OpenSearch Dashboards tab.
3. Copy or click the Service URI to open OpenSearch Dashboards in your browser.
4. Log in with the credentials provided in the Connection information section.

For more information, see OpenSearch Dashboards.

Connect to your service

Console

1. Go to the Overview page of your service in the Aiven Console.
2. Click Quick connect.
3. In the Connect window, select a dashboard or language to connect to your service.
4. Complete the actions in the window and click Done.

cURL

See Use Aiven for OpenSearch® with cURL for steps to connect using cURL.

Node.js

See Connect to OpenSearch® with Node.js to connect using Node.js.

Python

See Connect to OpenSearch® with Python to connect using Python.

Manage logs and monitor data

• Send logs: To send logs from Aiven services to OpenSearch, see Enable log integration.
• Monitor data: Set up Grafana for monitoring and alerts. See Integrate with Grafana®.

Search and aggregations with Aiven for OpenSearch

Aiven for OpenSearch® lets you write and execute search queries, as well as aggregate data using OpenSearch clients like Python and Node.js.

Write search queries

• Python: Learn how to write and run search queries on your Aiven for OpenSearch service using the Python OpenSearch client. For more information, see Search with OpenSearch® and Python.
• Node.js: Learn how to write and run search queries on your Aiven for OpenSearch service using the OpenSearch JavaScript client. For more information, see Search with OpenSearch® and Node.js.

Perform aggregations

• Metric aggregations: Calculate metrics such as average, minimum, maximum, percentiles, and cardinality on your Aiven for OpenSearch service. or more information, see Aggregations with OpenSearch® and Node.js.
• Bucket aggregations: Group data into buckets based on ranges, unique terms, or histograms. or more information, see Bucket aggregations with OpenSearch®.
• Pipeline aggregations: Combine results from multiple aggregations, such as calculating moving averages, to analyze trends. Explore examples in Pipeline aggregations with OpenSearch®.