Aiven now supports Apache Kafka version 2.4, allowing you to create a service on any of our supported clouds. Aiven is the first cloud service to provide Apache Kafka 2.4 as a fully managed service.
Apache Kafka 2.4 brings many new core features and improvements, along with enhancements to supporting services, like MirrorMaker, Kafka Connect and the REST endpoint. These, and others, are elaborated on in the release notes.
Apache Kafka 2.4 includes the following updates:
- KIP-382: MirrorMaker 2.0
- KIP-392: Allow consumers to fetch from closest replica
- KIP-440: Extend Connect Converter to support headers
- KIP-480: Sticky Partitioner
- KIP-455: Create an Administrative API for Replica Reassignment
MirrorMaker 2.0 improvements
Kafka 2.4 introduces MirrorMaker 2.0, which solves several problems related to legacy MirrorMaker installations that render them insufficient for backup and fail-over use cases. One of these problems: rebalancing itself can trigger further rebalances, due to latency spikes.
Along with this, partition configurations and migrations between mirrored clusters each introduce issues in multi-cluster environments. The new MirrorMaker, shipping with Kafka 2.4 is optimized for multi-cluster and cross-data-center deployments. MirrorMaker 2.0 is intended as a drop-in replacement for legacy instances.
NOTE: MirrorMaker 2.0 will be added later as an update to Aiven for Apache Kafka 2.4.
How to get Apache Kafka 2.4
When creating the service from Aiven console, simply select ‘2.4’ in the drop-down menu from the Kafka service box. You can also create a Kafka 2.4 service with the appropriate switch from the Aiven REST API, command line interface, or your Terraform tooling.
With Apache Kafka 2.4, you’ll get a range of new features plus security and authentication enhancements. For a comprehensive list of improvements coming in Apache Kafka 2.4, check out this post on the Apache Blog.