Hot and cold data with Apache Kafka, Tiered Storage, and Iceberg

Utilizing the true potential of data streaming is key to business success.

In this Data (R)evolution episode, we're joined by Josep Prat and Filip Yonov to dive into the transformative features of Apache Kafka and its evolving role in data architecture. They discuss the critical importance of collaboration and feedback in enhancing Kafka's capabilities, the future of "lake house" technology, exciting updates from the Open Source Program Office (OSPO), and the importance of Kafka's readiness to support evolving data formats—making it a backbone for modern data ecosystems.

Key takeaways:

  • Community collaboration and contribution are essential for the continuous improvement and testing of Apache Kafka's capabilities.
  • The evolution of Apache Kafka into a more versatile platform, combined with object storage and open table formats, can significantly enhance real-time data streaming, analytics, and the future of "lake house" technology.
  • Tiered storage in Kafka facilitates more efficient and cost-effective data management by decoupling storage from computing.

Podcast series: Data (R)Evolution

Topics around data, innovation and economics in a rapidly-changing cloud world.

Data drives innovation and revolutionizes the tech world. Data (R)evolution, powered by Aiven, explores the intersections of data, AI, innovation, and economics through the ever-evolving landscape of cloud technology. Led by hosts Francesco Tisiot and Janki Patel Westenberg, navigate this digital revolution in conversation with experts at the forefront of change.

Subscribe and never miss an episode

Watch the series

Hosts

Francesco Tisiot

Field CTO & Head of DevEx, Aiven

Francesco Tisiot, Field CTO & Head of DevEx at Aiven, is dedicated to helping customers achieve success through data-driven innovation. Francesco believes that technology should be an enabler, not a barrier, and works closely with clients to understand their unique needs and develop tailored solutions that deliver measurable results. A seasoned technologist with a proven track record, Francesco is a trusted advisor to organizations across industries.

Janki Patel Westenberg

Head of Product Growth and Marketing

Over the last 10 years, Janki has successfully led technical partnerships collaborating with Google Cloud and AWS to build database and caching solutions for gaming and media. Additionally, she has spearheaded GTM strategy partnered with AWS' product leadership for 19 AWS Analytics and Database services for multiple industries. Janki also is a proud alumna from University of Wisconsin-Madison (Go Badgers!)...fun fact, she studied Genetics and did research before moving to the cloud industry!

Guests

Josep Prat

Senior Engineering Director, Streaming Services, Aiven

Josep Prat is a distinguished leader in the open-source community, serving as the Senior Director for Streaming Services at Aiven. With more than a decade of deep involvement in open source, he is a maintainer of several key projects, most notably Apache Kafka. His leadership journey at Aiven also includes founding and heading its Open Source Program Office (OSPO), where he shaped the company's strategy for open-source engagement and sustainability.

Maja Schermuly

Solution Architect

Maja is a Solution Architect at Aiven, specializing in designing and implementing cloud data architectures. Her hands-on experience allows her to enable customers to build scalable solutions that are both practical and effective.