Create service integrations

Create service integrations between different Aiven services and move telemetry data using these integrations.

tip

For help with setting up a service integration or to request an integration type or endpoint not yet available, contact the support team

The following example shows you how to create and integrate these services:

• An Aiven for Apache Kafka® service that produces the telemetry data.
• An Aiven for PostgreSQL® service where the telemetry data is stored and can be queried.
• An Aiven for Grafana® service for the telemetry data.

Prerequisites

Console

• Access to the Aiven Console

Terraform

• Terraform installed
• A personal token

Create the integrations

Console

1. In the Aiven Console, create the 3 services: Aiven for Apache Kafka®, Aiven for PostgreSQL®, and Aiven for Grafana®.

2. On the Services page, open the PostgreSQL service.

3. Click Integrations.

4. Under Aiven services, click Grafana Metrics Dashboard.

5. Select Existing service and choose the Grafana service you created.

6. Click Enable.

7. To send metrics from the Apache Kafka service, under Aiven services select Receive Metrics.

8. Select Existing service and choose the Apache Kafka service you created.

9. Click Enable.

Terraform

1. Create a file named provider.tf and add the following:

   Loading...

2. For the services, create a file named services.tf and add the following:

   Loading...

3. To integrate the services, create a file named integrations.tf and add the following:

   Loading...

4. Create a file named variables.tf and add the following:

   Loading...

5. Create a file named terraform.tfvars and add values for your token and Aiven project.

To apply your Terraform configuration:

1. Initialize Terraform by running:

   terraform init

   The output is similar to the following:

   
   Initializing the backend...
   
   Initializing provider plugins...
   - Finding aiven/aiven versions matching ">= 4.0.0, < 5.0.0"...
   - Installing aiven/aiven v4.9.2...
   - Installed aiven/aiven v4.9.2
   ...
   Terraform has been successfully initialized!
   ...

2. To create an execution plan and preview the changes, run:

   terraform plan

3. To deploy your changes, run:

   terraform apply --auto-approve

To view the Apache Kafka metrics data in Grafana:

1. In the Aiven Console, go to your Grafana service.

2. In the Connection information section, click the Service URI to open Grafana.

3. Log in using the User and Password provided in the Connection information section.

4. On the dashboards page in Grafana, click the dashboard name to view it.

If you don't see a dashboard after logging in, search for Aiven Kafka - <KAFKA_SERVICE_NAME> - Resources in the Grafana console. This is a predefined dashboard automatically maintained by Aiven. Data can take a minute to appear on the dashboard after enabling the integrations. Refresh the view by reloading the page.

warning

Any modifications you make to the predefined dashboard are automatically overwritten by the system during updates.

You can create your own custom dashboards, or make a copy of this predefined dashboard to customize it. Don't use Aiven at the start of your dashboard names.