Prevent full disks

Ensure your Aiven for Apache Kafka® services run smoothly by preventing low disk space. The Aiven platform actively monitors disk usage, triggering notifications when it exceeds 90%.

If any node in the service surpasses the critical threshold of disk usage (more than 95%), the access control list (ACL) used to authorize API requests by Apache Kafka clients is updated on all nodes. This update prevents operations that can further increase disk usage, including:

• The Write and IdempotentWrite operations are used by clients to produce new messages.
• The CreateTopics operation creates new topics, each of which carries some overhead on disk.

When the disk space is insufficient and the ACL blocks write operations, you encounter an error. For example, when using the Python client for Apache Kafka, you might receive the following error message:

TopicAuthorizationFailedError: [Error 29] TopicAuthorizationFailedError: your-topic

Upgrade to a larger service plan

Console

To resolve disk space issues, you can upgrade to a larger service plan:

1. In the Aiven Console, select your project and choose your Aiven for Apache Kafka® service.
2. On the sidebar, Click Service settings.
3. In to Service plan section, click Actions > Manage additional storage.
4. In the Upgrade service storage screen, click Change plan choose the new service plan and tier or use the slider to adjust disk storage.

This deploys new nodes with increased disk space. After data migration from the old nodes to the new ones, disk usage returns to an acceptable level, and write operations are allowed again.

CLI

To upgrade your service plan using the Aiven CLI:

avn service update --project <PROJECT_NAME> --service_name <SERVICE_NAME> --plan <NEW_PLAN_NAME> --disk-space-gib <DISK_SPACE_GIB>

Parameters:

• <PROJECT_NAME>: The name of the project.
• <SERVICE_NAME>: The name of the Aiven for Apache service to update.
• --plan <NEW_PLAN_NAME>: The Aiven subscription plan name. Refer to avn_service_plan for available plans.
• --disk-space-gib <DISK_SPACE_GIB>: The total amount of disk space for data storage (in GiB).

Add additional storage space

For instructions on adding storage space, see Scale disk storage.

Delete one or more topics

Console

Free up disk space by deleting topics:

1. In the Aiven Console, select your project and choose your Aiven for Apache Kafka® service.

2. On the sidebar, click Topics.

3. To delete an existing topic, click the topic to remove.

   • In the Topic info screen, click Delete and confirm the deletion.
   • Alternatively, in the topic row, click the Actions > Delete topic.

CLI

To delete topics using the Aiven CLI:

avn service topic-delete --service_name SERVICE_NAME --topic TOPIC_NAME

Parameters:

• SERVICE_NAME: The name of your Aiven for Apache Kafka service.
• TOPIC_NAME: The name of the topic to delete.

API

To delete topics using the Aiven API:

curl -X DELETE \
  "https://api.aiven.io/v1/project/PROJECT_NAME/service/SERVICE_NAME/topic/TOPIC_NAME" \
  -H "Authorization: Bearer YOUR_API_TOKEN"

Parameters:

• PROJECT_NAME: The name of your project.
• SERVICE_NAME: The name of your Aiven you for Apache Kafka service.
• TOPIC_NAME: The name of the topic to delete.
• API_TOKEN: Your Aiven token.

Deleting topics frees up the disk space they used. The log cleaner process can take a few minutes to remove the associated data files from the disk. Once complete, the access control list (ACL) updates to allow write operations.

note

Admin access is required to perform this action.

Decrease retention time/size

To free up space without deleting a topic, consider reducing the retention time or size for one or more topics. If you know the age of the oldest messages in a topic, you can lower the retention time to make more space available. For more details, see how to change the retention period.