There are at least 3 ways you can buy Data Streaming in the cloud, one of them is Diskless BYOC. What are they? Where do they fail? Does BYOC fit for you?
Filip Yonov
|RSS FeedHead of Streaming Services
Subscribe for the latest news and insights on open source, Aiven offerings, and more.
Diskless Kafka splits storage from compute, delegating replication to cheap object storage and turning Apache Kafka® Brokers into a stateless compute layer. It’s 100% Kafka, and 80% cheaper.
But in the cloud, a cheaper underlying technology does not always mean you pay less. The cost varies significantly depending on the deployment model - SaaS or BYOC. In this article, we will learn why:
1. Fully-managed solutions do not always reduce costs as much as you think
2. Bring Your Own Cloud (BYOC) fits Diskless perfectly
Over the past year, as we built Diskless Kafka, the question we have received most consistently from Kafka operators has been “What exactly is BYOC—and is it worth the added complexity?”
Short answer: yes, if you're operating at scale.
Diskless BYOC nails the three E’s:
If you're pushing gigabyte scale workloads, have a Kafka bill north of $100k/year, or need to prove compliance boundaries without DIY, Diskless BYOC is the only Kafka architecture that hits all three. Diskless runs inside your cloud account, you own the IAM, VPCs, and audit trail. Data stays within the perimeter; cross-account networking disappears. The best thing? All cost savings are directly passed down to you.
We’ll walk through the common pitfalls, the cost model, and when to switch. Let’s dive in.
If you write code for a living you’ve probably heard of BYOC—but it’s often lumped together with “serverless,” “dedicated,” or plain-old “managed” and left to fend for itself. No wonder there’s confusion. To clear the fog, let’s do a lightning tour of the three ways you can consume Kafka today without doing it yourself.
TL;DR: You book a room in a hotel
You rent a handful of Kafka topics on an oversubscribed mega-cluster.
TL;DR: You rent the whole house.
You rent an entire set of VMs or containers in a vendor's account. Full tenancy, full SLA.
TL;DR: You own the house.
You only rent the code, but it runs inside your AWS/GCP/Azure account—next to your producers and consumers.
For more than a decade, the industry playbook has been: treat Kafka like an API, not a Cluster. You punch in a credit card, pick a region, dial the SLA slider, and—poof—topics or if you have larger workloads, clusters appear.
But the moment before you click “Create” you must ask the provocative question that every architect eventually faces: Do you want your Kafka easy…or do you want control and safety?
Vendors do their best to guide you (we do too!) —“start small and graduate later,” “just pay for what you stream”—but when the firehose hits a few MB/s and the data runs critical business processes, the math (and pricing incentive) tilts toward a stable dedicated cluster almost every time. After all, you rarely wake-up on a random Tuesday morning asking for “some Kafka in your life”, Data Streaming is an elite discipline which mandates careful planning.
Which raises a puzzle: if the answer is always “get a dedicated Kafka cluster”, why are we even talking about BYOC? After all, it’s still the same managed service—just deployed in your cloud account instead of the vendor’s.
You guessed it: the cloud is hiding something. Stick around; we’re about to pull the curtain on that mystery.
Usage-based billing sounds perfect for Kafka—pay only for what you write, store, and read. But, as we discussed, the moment Kafka traffic goes from hobby to heavy, you quickly realise you’re feeding two meters at once:
Ask yourself: When you sign-up for “fully-managed” streaming, how many times does that byte get resold before it lands in your consumer?
Here’s the real kicker: the priciest part of “Kafka” isn’t Kafka. It's the data moving within and around Kafka - the swarm of producers, consumers, connectors, and ETL jobs slinging data in and out. As we discussed in previous blogs, every hop lights up a data-transfer tollbooth.
If you want to understand Kafka’s SaaS pricing, there’s an important separation to make between cloud costs and vendor markups. Most vendors target ≈ 80% gross margin. Translating that to Kafka means that If a new feature burns $1 of cloud COGS—compute, storage, network—the list price must be $5. $1 is only 20% of $5; the other 80 % funds salaries, support, R&D, and profit.
Now layer it on Kafka’s physics:
A per-gigabyte transfer price that looks tidy on paper is really cloud tax × vendor multiplier × fan-out factor.
But nothing gives a better picture than a real-world example: let’s watch the “pay-as-you-go” model crack along the Produce → Consume path for a real-world 500 MB/s Kafka.
A typical usage-based SaaS model says “writes cost $0.05 per GB,” so you budget for five cents. What the price sheet glosses over is the extra $0.01 per GB AWS charges the moment your producers push bytes out of your VPC and into the vendor’s public endpoint. Remember - one byte, two cash registers.
Why is that penny so stubborn? Because most default deployments connect over a public IP; wiring private IPs across VPCs means PrivateLink, CIDRs, Terraform—hardly a “quick-start.” And remember: AWS still applies the cross-AZ tax even when the traffic never leaves the AZ as long as it rides that public hop.
That unaccounted costs changes nothing about durability, retention, or replication of your Kafka; it’s just the toll for letting bytes cross the street into your vendor’s account. And because most “quick-start” setups ride a public IP—even inside the same AZ—the meter never stops.
Kafka’s super-power is read amplification: most use-cases expect at least three consumer groups for every producer. That 3× multiplier means our 500 MB/s write stream becomes 1.5 GB/s on the way out.
Before we deep dive in the wild-west of Networks, Kafka storage hurts too! Say you have 0.05 / GB-month for a single copy of the log. Kafka keeps three. So the true bite is $0.24 / GB-month—about 10× the price of parking the same data in S3.
Add it up and a workload that looks like “$0.05/GB, no surprises” snowballs into $647k per year in cross-account network charges alone. So the real question is: what if we could erase those toll booths—keeping every byte inside your own VPC—and finally unleash the pent-up demand for truly massive, fully managed streaming? That’s precisely the gap BYOC is built to close.
In AWS you’ll even pay the $0.02/GB cross-AZ tax inside the same AZ if the traffic is over a public IP - which is the default account to account set-up! You discover the cross-AZ toll hiding in your bill and think, “Easy—let’s switch to private networking.” But, is it though?
First stop is VPC peering. It’s free on paper—until you realise every VPC must have a non-overlapping CIDR block. One duplicate subnet and the peering fails, which means renumbering workloads that have been humming for years. Add two more VPCs and the peerings explode into a full mesh; suddenly you’re mapping routes like an air-traffic controller - do you want to do this with a third party vendor?
You pivot to a Transit Gateway—clean hub-and-spoke topology, far less subnet juggling. Then the first invoice lands: AWS charges the same $0.02 per GB at the gateway itself plus a fixed hourly rate, effectively doubling the fee you were trying to erase. Now you’re paying the toll and running the toll plaza.
Next option: PrivateLink. Secure, one-way, and blissfully oblivious to overlapping CIDRs. But it isn’t quite free either—about $0.01 per GB—and Kafka’s brokers now advertise odd DNS names that every producer and consumer must learn. Each new AZ needs its own endpoint, each endpoint its own quota of IAM, TLS, and Terraform plumbing. The “managed” service now comes with a 30-page network run-book. Fallback? Keep the public endpoint and rely on NAT. Cost drops, security teams panic, and of course you’re right back to paying cross-AZ rates for traffic that never left the building.
In short, every escape hatch either reinstates the toll in a different guise or replaces it with weeks of network archaeology. So much for drop-in convenience. Shall we look at BYOC now?
(A three-question stress test for whether BYOC is really your move.)
Before we declare BYOC the winner, let’s take a step back. A handful of security‑sensitive customers asked Aiven to run Kafka inside their own cloud accounts without losing managed‑service uptime in 2018 (We built the first BYOC — custom account wiring, live‑incident hooks, zero‑downtime version upgrades— and have expanded it ever since.)
A BYOC cluster looks almost identical to one running in our VPC, yet it exists for a specific set of pain points. Decide if those pain points are yours by running through three brutally honest questions:
E-commerce keeps carts on-shore, hospitals quarantine PHI, telcos hide keys in hardware vaults—yet all that rigor crumbles if a single zero-day breaches the SaaS provider’s VPC where your data plane lives. This is a big deal because a single breach, can endanger your entire Kafka setup.
BYOC shrink-wraps Kafka inside your security boundary: your keys, IAM, flow logs, kill switch. The vendor still handles patches and scaling, but any blast radius stops at your VPC wall. New law? New exploit? Non-issue—because the data never left the building. BYOC is the only way to lock down Kafka’s data plane without going back to DIY toil.
Remember the math from our 500 MB/s example? That stream pushes 1.3 PB IN and 4 PB OUT every month. Because the bytes hop from your VPC to the vendor’s over a public IP, AWS tallies them as cross-AZ traffic—$0.01/GB on your side, another $0.01/GB on theirs. That’s $53k every month—$647k a year—in tolls no pricing page warns you about.
Stanislav Kozlovski did a great cost anatomy post which powerfully visualizes the problem. There is also a lesser known piece by Redpanda on the same topic.
AWS will happily sell you PrivateLink to knock the toll down to ~$0.01/GB + a monthly fee, but that’s still a six-figure line item with zero business value. BYOC vaporizes the toll entirely—producers, brokers, and consumers live inside the same account, chatting over a free network. That’s the superpower of BYOC: it turns the physics of cloud bandwidth into a rounding error on your cloud bill.
Bonus: those bytes now count toward your own compute/storage discounts instead of padding your vendor’s bill. Which leads to..
High-throughput streaming is an elite sport. Sustaining GB/s isn’t “turn on a feature”—it’s an entire engineering culture, and it usually drags seven-figure cloud commitments behind it. Your FinOps team already lands platinum-tier discounts:
That kind of structure comes with multi-million dollar commitments— 20% off EC2, 30% off S3, 80% off Networking, maybe more. Every VM you rent in a vendor’s VPC burns those discounts to ash. Move the cluster into your own account and the same bytes now count toward your rebate. You keep the managed-service brain, but the bulk-buyer coupon stays in your wallet.
Simply put: BYOC doesn’t lower the list price; it leverages the discount you already have.
If your Kafka is small i.e. 5-10 MB/s, your cloud bill modest, and your compliance team mellow, classic SaaS is perfect easy wins, zero plumbing.
But the moment the meter tips, discounts kick in, or regulators circle, you hit the fork in the road: Do you want Kafka that stays easy—or Kafka you can control?
BYOC exists so you can finally answer, “How about both?”
(spoiler: the “mystery” is mostly smoke)
We’ve shown how consumption pricing collapses under scale—but BYOC stickers can still look like stage-magic. Surely there must be some gotchas?
Go price-shopping for BYOC Kafka and you’ll find a desert: Aiven and WarpStream are the lone booths with a sticker on the window. Everyone else treats BYOC like the secret menu at a restaurant—“Ask the waiter, he’ll whisper a number.” The irony? The heavy lift already shows up on your cloud bill, not ours. All a vendor really needs to charge for is the stewardship: upgrades, monitoring and the four nines.
Because a “management fee” looks like pure margin on a spreadsheet—and margin invites haggling. Easier to keep the line item invisible. We went the other way: Aiven’s BYOC Cost Calculator breaks the price down—VM class, storage tier, support overhead—so you can audit it quickly and decide for yourself.
Similar to consumption pricing in SaaS, lately we’ve seen two anti-patterns in BYOC pricing:
Both feel innovative until you look closer:
High-scale BYOC customers don’t want a carnival of meters; they want a partnership: a clear monthly fee, a volume-based annual commit, and zero data leaving the house. Anything else is a step backward disguised as progress.
BYOC pricing doesn’t need to be mystical, at Aiven we believe it should be boring. Publish the fee, show the math, and let customers validate the TCO against their own cloud discounts. The moment you do, the “mystery” evaporates—and the real conversation can start.
We joke internally that “Diskless BYOC is a $100K-and-up club.” Not because we’re price-gouging, but because the very need for BYOC is a litmus test:
When all three lights are flashing, that six-figure line item quickly looks like the cheapest decision in the room—and the only way to keep streaming simple and under your control.
Ready to see if you’re in the $100k-and-up club? Try the Cost Calculator now and get an instant, line-item comparison.
Happy streaming 🚀
