Aiven for Apache Kafka^® delivers low latency and scale for Xfinity Home

Overview

Comcast Corporation (Nasdaq: CMCSA) is a global media and technology company with two primary businesses, Comcast Cable and NBCUniversal. Well known for its residential video, high-speed internet, and phone services, Comcast Cable also provides wireless security and automation services under Xfinity Home.

Ultimately, the aim of Xfinity Home is to create a digital home for its customers, providing a single experience that connects and manages all home security and automation needs by integrating some of the best products and devices.

Their Business Opportunities Manager, Andrew Gerhold, explains, “We combine our Xfinity Home and XFi experiences into our Digital Home Ecosystem to secure our customers’ homes and enable IoT concept across all stacks of our devices to deliver a truly smart home! Specifically, our engineers are working to enable our customers to easily control their connected devices, making full use of our Xfinity platforms.”

The challenge

Building the Digital Home Ecosystem requires a team that specializes in building high-performance, reliable back-end systems and server-side APIs, creating the interconnectivity fabric of the platform and enabling the provisioning of a revolutionary class of services to millions of internet users.

Tens of millions of customers means hundreds of millions of networked devices—and an enormous amount of data. This data is used to deliver personalized content but also to ensure a resilient product and improve customer care.

And as if the sheer amount of data weren’t enough of a challenge, Comcast’s diverse customer base provides an additional one. Not only are they spread across many regions, they use dozens of different devices that are connected to hundreds of access networks at different times.

This results in latency that is averaged over millions of customers all under constantly changing conditions. As Gerhold explains, “Consequently, the complexity and volatility inherent in the real-world conditions under which end users expect IoT devices to perform must be matched by an equally performant solution.”

The solution

While latency is a key metric, scalability is also essential. After prototyping and testing a few inhouse and external solutions, Comcast decided to move forward with Kafka because it has better throughput, built-in partitioning, replication, and fault tolerance, which makes it a good solution for large scale message processing applications.

In the team’s experience, messaging uses are often comparatively low-throughput but may require low end-to-end latency and often depend on the strong durability guarantees Kafka provides.

Kafka has lots of benefits, but it is difficult to manage and takes resources away from bottom-line projects. This is why they vetted providers according to very specific requirements. “The cluster must maintain high availability and must be performant 98% of the time under 50ms end-to-end,” Gerhold explains. After a thorough vetting process, they chose Aiven for Kafka and Aiven’s metrics integration.

The outcome

Ultimately, Aiven for Kafka was able to meet the latency and scalability requirements while outperforming the other competitors. Their VP of Engineering, Adam Hertz puts it simply: “At Comcast, we needed a robust managed Kafka solution for some of our most critical workflows. Based on our evaluation, Aiven clearly offers superior cost, support and performance.”

Comcast’s choice of Aiven is a testament to its focus on providing easy-to-use, secure, and performant solutions for production workloads, which require such features to be provided dependably.