The Kafka cost estimator reflects a real-world deployment across three availability zones (AZs). It includes key features like Tiered Storage and Fetch-from-Follower, SSD-backed brokers with built-in capacity headroom, and realistic cloud pricing for compute, storage, and cross-AZ traffic.
When diskless topics are selected, the model also accounts for the lower operational effort required to run these clusters.
1. High availability and replication
All Kafka clusters modeled in the calculator are spread across three AZs. Each topic uses a replication factor of 3, ensuring durability and availability.
This replication level is fixed in the estimator and matches Apache Kafka’s default recommendation for production environments.
2. Kafka features enabled by default
The estimator assumes two key features are always on:
- Tiered Storage (KIP-405)Recent data (up to 12 hours) is stored on local SSDs. Older data is offloaded to object storage such as Amazon S3 or Google Cloud Storage.
- Fetch-from-Follower (KIP-392)Consumers read from in-zone brokers, keeping 100% of traffic local to each AZ and reducing cross-AZ costs.
3. Disk and capacity guardrails
To reflect realistic operational behavior, the estimator includes resource usage limits for each broker:
Constraint | Reason |
|---|
SSD-only storage | HDDs increase tail latency; SSDs match Aiven’s Kafka fleet |
≤ 40% disk usage per broker | Leaves headroom for rebalancing and unexpected traffic spikes |
≤ 80% CPU and network utilization | Reduces jitter and prevents resource throttling |
Maximum disk size: 16 TiB | Larger EBS volumes slow down restarts significantly on AWS |
If a workload can tolerate HDD latency, the calculator may favor Diskless Topics. Offloading data to remote storage removes the local disk I/O bottleneck.
4. Cluster sizing and partition limits
To avoid excessive partition reassignments and reduce mean time to recovery (MTTR), the calculator applies Kafka community sizing guidance:
- Up to 60 brokers per cluster
- Up to 4,000 partitions per broker
- Up to 200,000 partitions per cluster
- Up to 10,000 client connections per broker
When these thresholds are exceeded, the model assumes a second cluster is created.
5. Cross-AZ network pricing
The estimator uses actual cloud provider pricing for cross-AZ data transfer:
Cloud provider | Cross-AZ cost (per GiB) |
|---|
AWS | $0.02 (bidirectional) |
Google Cloud | $0.01 |
Azure | $0.00 (within region only) |
Cross-region traffic is not included by default. To model inter-region mirroring, enable the corresponding option in the calculator.
6. Operational effort
The estimator includes operational staffing assumptions based on Aiven’s internal telemetry:
Cluster type | Staffing estimate (FTE per 100 MiB/s sustained ingest) |
|---|
Classic Kafka | 0.5 FTE |
Diskless Kafka | 0.125 FTE |
Clusters using only Diskless Topics require less manual intervention. By offloading data to object storage, Tiered Storage removes the need for local disk management. This simplification also reduces the impact of incidents. The staffing estimates used in the calculator are intentionally conservative compared to public total cost of ownership (TCO) models.
