Configure audit logging for Aiven for Apache Kafka®

Turn audit logging on for your Aiven for Apache Kafka® service, change what it records, and manage audit log volume.

For what audit logging captures and its limitations, see Audit logging for Aiven for Apache Kafka®.

important

Before you turn on audit logging, note the following:

• Turning audit logging on or changing its settings restarts the Kafka brokers in your service one at a time. Make these changes during a maintenance window or a period of low traffic.
• After you turn on audit logging, you cannot remove audit logging settings or turn off audit logging yourself. To turn off audit logging, contact Aiven support.

Prerequisites

To configure audit logging, you need one of the following project roles or permissions:

• admin: Full access to services in the project.
• operator: Full service management access.
• project:services:write: Broad services write access.
• service:configuration:write: Least-privilege access for changing service configuration.

The developer and read_only roles cannot configure audit logging.

Enable audit logging

To enable audit logging, add at least one kafka.audit_log setting to your service configuration. Any setting you add must have a valid value.

Aiven Console

1. In the Aiven Console, open your Aiven for Apache Kafka service.
2. Click Service settings.
3. In the Advanced configuration section, click Configure.
4. Click Add configuration options and enter audit to find the audit logging settings.
5. Add kafka.audit_log.record_type and select user_operations.
6. Optional: Add other audit logging settings and set their values.
7. Click Save configuration.

Aiven CLI

Set kafka.audit_log.record_type with the avn service update command and the -c flag:

avn service update SERVICE_NAME \
  --project PROJECT_NAME \
  -c kafka.audit_log.record_type=user_operations

Replace SERVICE_NAME and PROJECT_NAME with your service and project names.

Aiven API

Send a PUT request to the service update endpoint:

curl -s -X PUT \
  --url "https://api.aiven.io/v1/project/PROJECT_NAME/service/SERVICE_NAME" \
  -H "Authorization: Bearer TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "user_config": {
      "kafka": {
        "audit_log": {
          "record_type": "user_operations"
        }
      }
    }
  }'

Replace PROJECT_NAME, SERVICE_NAME, and TOKEN with your project name, service name, and authentication token.

Terraform

Add an audit_log block to the kafka_user_config block of your aiven_kafka resource:

resource "aiven_kafka" "example_kafka" {
  project      = var.project_name
  cloud_name   = "google-europe-west1"
  plan         = "business-4"
  service_name = "example-kafka"

  kafka_user_config {
    kafka {
      audit_log {
        record_type = "user_operations"
      }
    }
  }
}

Audit logging settings

Use these advanced configuration settings to customize audit logging. In the service configuration, add these settings under kafka.audit_log, for example kafka.audit_log.record_type.

Setting	Type	Default	Description
record_type	string	user_operations	The type of activity to record. Use user_operations for detailed operation entries, or user_activity to record only that a Kafka user was active.
aggregation_period_sec	integer	300	How long, in seconds, to group entries before writing them to the service logs. A higher value produces fewer, larger entries. Accepts a value from 1 to 1800.
include_denials	boolean	false	Whether to include denied authorization attempts in audit log entries. When false, audit log entries include only allowed operations.
group_by	string	user_and_ip	How to group entries: by Kafka user only (user), or by Kafka user and IP address (user_and_ip). Applies only when record_type is user_operations.

Change audit logging settings

To change what audit logging records, set new values for the kafka.audit_log settings with any of the preceding methods. Services that already use audit logging keep their current settings until you change them.

View audit logs

Audit entries appear in the service logs with the AUDIT: prefix. To view them, use one of the following methods:

• In the Aiven Console, open your service and click Logs.

• With the Aiven CLI, run:

  avn service logs SERVICE_NAME \
    --project PROJECT_NAME \
    | grep AUDIT:

• Send the service logs to another system through a log integration.

Manage audit log volume

Audit logging can produce many log entries. To manage the volume:

• Set group_by to user instead of user_and_ip to combine a Kafka user's activity across IP addresses.
• Increase aggregation_period_sec to group entries over a longer time window.
• Keep include_denials set to false unless you need denied attempts in audit log entries.
• Use user_activity instead of user_operations when you only need to know which Kafka users were active.

Related pages

• Audit logging for Aiven for Apache Kafka®
• Advanced parameters for Aiven for Apache Kafka®
• Integrate service logs into an Apache Kafka® topic
• Monitor and alert logs for denied ACL