Create service integrations
Create service integrations between different Aiven services and move telemetry data using these integrations.
For help with setting up a service integration or to request an integration type or endpoint not yet available, contact the support team
The following example shows you how to create and integrate these services:
- An Aiven for Apache Kafka® service that produces the telemetry data.
- An Aiven for PostgreSQL® service where the telemetry data is stored and can be queried.
- An Aiven for Grafana® service for the telemetry data.
Prerequisites
- Console
- Terraform
- Access to the Aiven Console
Create the integrations
- Console
- Terraform
-
In the Aiven Console, create the 3 services: Aiven for Apache Kafka®, Aiven for PostgreSQL®, and Aiven for Grafana®.
-
On the Services page, open the PostgreSQL service.
-
Click Integrations.
-
Under Aiven services, click Grafana Metrics Dashboard.
-
Select Existing service and choose the Grafana service you created.
-
Click Enable.
-
To send metrics from the Apache Kafka service, under Aiven services select Receive Metrics.
-
Select Existing service and choose the Apache Kafka service you created.
-
Click Enable.
-
Create a file named
provider.tf
and add the following:Loading...
-
For the services, create a file named
services.tf
and add the following:Loading...
-
To integrate the services, create a file named
integrations.tf
and add the following:Loading...
-
Create a file named
variables.tf
and add the following:Loading...
-
Create a file named
terraform.tfvars
and add values for your token and Aiven project.
To apply your Terraform configuration:
-
Initialize Terraform by running:
terraform init
The output is similar to the following:
Initializing the backend...
Initializing provider plugins...
- Finding aiven/aiven versions matching ">= 4.0.0, < 5.0.0"...
- Installing aiven/aiven v4.9.2...
- Installed aiven/aiven v4.9.2
...
Terraform has been successfully initialized!
... -
To create an execution plan and preview the changes, run:
terraform plan
-
To deploy your changes, run:
terraform apply --auto-approve
To view the Apache Kafka metrics data in Grafana:
-
In the Aiven Console, go to your Grafana service.
-
In the Connection information section, click the Service URI to open Grafana.
-
Log in using the User and Password provided in the Connection information section.
-
On the dashboards page in Grafana, click the dashboard name to view it.
If you don't see a dashboard after logging in, search for
Aiven Kafka - <KAFKA_SERVICE_NAME> - Resources
in the Grafana console.
This is a predefined dashboard automatically maintained by Aiven. Data can take a minute
to appear on the dashboard after enabling the integrations. Refresh the view by reloading
the page.
Any modifications you make to the predefined dashboard are automatically overwritten by the system during updates.
You can create your own custom dashboards, or make a copy of this predefined dashboard
to customize it. Don't use Aiven
at the start of your dashboard names.