The new Apache Kafka version 2.8, announced on April 19, 2021, comes with a number of enhancements across the board, including a preview of being able to run Kafka without Zookeeper.
Apache Kafka without ZooKeeper
KIP-500: Replace ZooKeeper with a Self-Managed Metadata Quorum
Exciting news, removing the ZooKeeper dependency is moving forward, and now it is possible to run Kafka without ZooKeeper. The new consensus based on the RAFT protocol, KRaft, is now available. Note however, that the feature is not production ready yet and should only be used for testing. But it is already showing promising performance improvements.
Exactly Once delivery
KIP-679: Producer will enable the strongest delivery guarantee by default
Starting with the version 2.8, the Kafka producers will now have “exactly once” delivery semantics enabled by default. Transaction producers have been available since version 0.11, and provide much better semantics for application development with a very small performance hit.
KIP-612: Ability to Limit Connection Creation Rate on Brokers
Version 2.8 extends the throttling capabilities introduced in 2.7. Now it is possible to throttle the number of connections created per IP, which allows more fine grained control to prevent performance degradation when up-scaling or deploying an application with a high number of nodes.
Improved leader election performance for clusters with thousands of partitions
A small bug fix with a large effect. This one liner halved the time to process
some leader election messages.
The list above includes only examples of the most important enhancements in this release. To see the full list please visit: https://archive.apache.org/dist/kafka/2.8.0/RELEASE_NOTES.html
How to get started with Aiven for Apache Kafka 2.8
You can either launch a brand new cluster in the Aiven console or conduct a no-downtime upgrade to the latest version.
Not using Aiven services yet? Sign up now for your free trial at https://console.aiven.io/signup!
Feb 3, 2022
Database migration with Apache Kafka® and Apache Kafka® Connect
Find out how to use Apache Kafka® to migrate across database technologies while keeping the target continually in sync with the source.
Nov 25, 2021
Aiven supports Apache Kafka® 3.0
Apache Kafka® version 3.0 was announced on September 21, 2021, and Aiven already supports it. Find out about the enhancements across the board.
Nov 3, 2021
Optimizing data streaming pipelines: a panel recap
Apache Flink and Apache Kafka go together like foo and bar. Read or listen to panelists Olena Babenko, Francesco Tisiot, and Gyula Fora explain their take.
Subscribe to the Aiven newsletter
All things open source, plus our product updates and news in a monthly newsletter.