What 16,808 Kafka Clusters Tell Us About Data Streaming

Half a year ago, we launched a free tier cloud Kafka.

We have 16,808 clusters so we got curious: what are these builders telling us about the state of Apache Kafka?

The headlines this quarter suggest Kafka is dying because the streaming market is consolidating. At Aiven we see the opposite. Kafka is not shrinking. It is spreading outward from enterprise platform teams into the hands of individual builders.

We are now seeing >200 new Kafka clusters created per day on the free tier. They are solo developers spinning up Kafka to test an idea before anyone gives them permission or their project requires it.

I believe the health of open source Apache Kafka should not be measured by top-down decisions like how many Fortune 500s run it. It's measured bottom-up: by asking whether the individual developer still reaches for Kafka before they absolutely have to.

16,808 Kafkas under a microscope

About 19% of free-tier clusters showed Kafka client activity during the period we measured. Only 6% had more than three topics.

The lazy read is that most of these clusters never became serious Kafka usage.

I think that misses the point. Free-tier behavior is not production - It is exploration. The signal is not how many clusters immediately became mature systems. It’s what developers reach for when cloud Kafka becomes free to try.

In 2026, starting with Kafka is still too hard. Not technically hard. Psychologically hard.

Kafka comes with a reputation as a project consisting of many interdepending parts and ideas: brokers, partitions, replication, schema registry, connectors, processing, quotas, costs, incident response. Even when all of that is managed for you, the idea of committing to a Kafka cluster still feels heavy.

Our free tier changed that. It gives Kafka-curious developers a place to go before they are Kafka-committed. Before there is a platform team or even a budget. They can spin up the real thing, send a message with a single click, connect a client, and decide whether Kafka belongs in the idea. The more forward-looking ones have Claude do all of that for them.

That is why the shape of usage matters more than the totals. Most free clusters will never become production clusters. That is fine.

> Here's what the curious actually do.

They create a cluster. They produce a few messages. They test some features (52% reached for our free data generator). They connect to a service. And sometimes they leave, but as we saw in our data they often come back. Sometimes the experiment becomes the first real stream in the system.

Our 16,389 free Kafka cluster users mostly fall into two groups.

The first are engineers - often Java, Scala, or Python developers, and often students - building applications that produce and consume from Kafka topics. The second are data engineers and data scientists moving CDC events from databases or running analytics on data while it is still in motion.

But the most revealing signal is what they create next. Of those Kafka free-tier accounts, 7,051 also created an Aiven MySQL service. Another 3,514 created PostgreSQL. Free-tier Kafka users are also one of the most significant sources of new ClickHouse services on our platform.

That tells us who these users really are. Most are not streaming purists designing perfect event-driven architectures. They are not infrastructure architects building the canonical Kafka-to-search pipeline. They are builders. They come in because Kafka is the name they recognize, but what they want is a place to build a working application: Kafka for events, MySQL or PostgreSQL for state, and ClickHouse for analytics.

The dominant pattern is not log analytics. It is something more ordinary, and probably more important: CRUD apps with event-driven backends.

We had more than a thousand clusters hit the 250 KB/s throughput ceiling we had to put on the free tier. A thousand developers, on a tier built for hobbyists, found the wall. That number startled me more than the total.

The same data tells two stories at once:

• The majority of users want Kafka to be tiny for learning, for hobby projects, for the messy first prototype. Nobody starts by testing at 50,000 messages a second. They need a few dozen so they can fit the whole flow in their head.
• A meaningful minority wants to immediately make it sweat. Turns out the streaming market isn't a bell curve. It's a reverse J shape. A handful of giants at one end. A long tail of sub-1 MB/s builders at the other with the middle thinned out.

Almost nobody is designing Kafka for the first serious stream. We think that's wrong.

At Aiven, we proudly bet on the success of developers. The developers who do not need a giant broker fleet, but do need real Kafka.

Here's the problem: 80% of Kafka clusters run under 1 MB/s, but the industry prices them like they're preparing to become Netflix by Thursday.

That's absurd.

A streaming app doesn't need a giant broker fleet or enterprise-scale architecture on day one. It needs real Kafka, real clients, Schema Registry, REST access, observability, connectors - and a price that matches the workload.

So we are proud to announce the next step in that bet: the Aiven Developer Kafka cluster.

One Small Step for Aiven, One Giant Leap for Small Kafkas

Given the overwhelming usage of the Free Tier, we're taking the next big step for small Kafkas.

The fix isn't a faster Kafka. It's better economics and ergonomics on the low end. We started with one question: how do we deliver maximum value at the lowest price point in the industry? The answer crystallized into a simple requirement — serious Kafka at the cost of a Netflix subscription.

Here's what you get for $35/month:

• 1 MB/s data in/out on a self-balancing cluster
• 20 topics, up to 100 partitions
• Schema Registry and REST proxy included
• Production Prometheus integration included
• 99.9% SLA

On top of the $35 base, add any of our 50+ managed connectors for $20 each, plus configurable storage starting at $10. Zero gotchas. Each connector is capped at 200 KB/s .Intentionally modest, but a strong fit for development, prototypes, demos, and early production validation.

Ok Claude, build me a Kafka. Make no mistakes.

The SaaSpocalypse is overstated. People don't actually want to maintain their own software. I tried rebuilding Kafka over a weekend and reached the same conclusion everyone reaches: complex software isn't a product you install. It's a garden you tend. The thing you pay for in "as-a-service" is the tending.

Even OpenAI runs on Apache Kafka. If the frontier labs aren't building their own, why would we?

That being said, AI agents can still be massively helpful in your Kafka journey. Getting started can be one line:

Loading code...

Then:

> create aiven kafka

That's it. Go build something cool.