Configure the ENV secret provider

Configure and use the ENV secret provider in Aiven for Apache Kafka® Connect services.

Prerequisites

• Aiven for Apache Kafka service with Apache Kafka Connect set up and running.
• Aiven CLI.
• Aiven Terraform Provider installed.

note

The ENV secret provider is not yet available in the Aiven Console.

Configure the secret provider

Configure the ENV secret provider in your Aiven for Apache Kafka Connect service to store and reference secrets in user_config.

API

Use the ServiceUpdate API to update your service configuration. Add the ENV secret provider configuration to user_config with the following API request:

curl --request PUT \
  --url https://api.aiven.io/v1/project/PROJECT_NAME/service/SERVICE_NAME \
  --header 'Authorization: Bearer AIVEN_API_TOKEN' \
  --header 'Content-Type: application/json' \
  --data '{
    "user_config": {
      "secret_providers": [
        {
          "name": "db_credentials",
          "env": {
            "secrets": {
              "db_password": "DB_PASSWORD_VALUE"
            }
          }
        }
      ]
    }
  }'

Parameters:

• name: Name of the secret provider, for example db_credentials.
• env.secrets: Map of secret keys and values stored in user_config.
• db_password: Secret key that you use later in connector configuration.

Terraform

Configure the ENV secret provider using Terraform. Add this configuration to your main.tf file, or create a dedicated file for secret providers:

resource "aiven_kafka_connect" "kafka_connect" {
  project      = var.project_name
  cloud_name   = var.cloud_name
  plan         = var.plan
  service_name = var.service_name

  kafka_connect_user_config {
    secret_providers {
      name = "db_credentials"
      env {
        secrets = {
          db_password = var.db_password
        }
      }
    }
  }
}

Parameters:

• name: Name of the secret provider, for example db_credentials.
• env.secrets: Map of secret keys and values.
• db_password: Terraform variable containing the secret value.

CLI

Add the ENV secret provider using the Aiven CLI:

avn service update SERVICE_NAME \
  -c secret_providers='[
    {
      "name": "db_credentials",
      "env": {
        "secrets": {
          "db_password": "DB_PASSWORD_VALUE"
        }
      }
    }
  ]'

Parameters:

• SERVICE_NAME: Name of your Aiven for Apache Kafka service.
• name: Name of the secret provider, for example db_credentials.
• env.secrets: Map of secret keys and values.

Reference secrets in connector configurations

Reference secrets in connector configuration values using the provider name and secret key. Use the syntax ${PROVIDER_NAME:SECRET_KEY}.

Example values:

• Provider name: db_credentials
• Secret key: db_password
• Secret reference: ${db_credentials:db_password}

JDBC sink connector

Example JDBC sink connector configuration that references a secret from the ENV secret provider.

{
  "name": "jdbc-sink-connector",
  "connector.class": "io.aiven.connect.jdbc.JdbcSinkConnector",
  "connection.url": "jdbc:postgresql://DB_HOST:5432/DB_NAME?user=DB_USER&password=${db_credentials:db_password}&ssl=require",
  "topics": "YOUR_TOPIC",
  "auto.create": true
}

JDBC source connector

Example JDBC source connector configuration that references a secret from the ENV secret provider.

{
  "name": "jdbc-source-connector",
  "connector.class": "io.aiven.connect.jdbc.JdbcSourceConnector",
  "connection.url": "jdbc:postgresql://DB_HOST:5432/DB_NAME?ssl=require",
  "connection.user": "DB_USER",
  "connection.password": "${db_credentials:db_password}",
  "mode": "incrementing",
  "incrementing.column.name": "id",
  "table.whitelist": "YOUR_TABLE",
  "topic.prefix": "jdbc_"
}

Security behavior

The ENV secret provider stores secrets in encrypted form at rest. The service decrypts secrets in memory only when a connector resolves them at runtime.