Hot and cold data with Apache Kafka, Tiered Storage, and Iceberg

Utilizing the true potential of data streaming is key to business success.

In this Data (R)evolution episode, we're joined by Josep Prat and Filip Yonov to dive into the transformative features of Apache Kafka and its evolving role in data architecture. They discuss the critical importance of collaboration and feedback in enhancing Kafka's capabilities, the future of "lake house" technology, exciting updates from the Open Source Program Office (OSPO), and the importance of Kafka's readiness to support evolving data formats—making it a backbone for modern data ecosystems.

Key takeaways:

  • Community collaboration and contribution are essential for the continuous improvement and testing of Apache Kafka's capabilities.
  • The evolution of Apache Kafka into a more versatile platform, combined with object storage and open table formats, can significantly enhance real-time data streaming, analytics, and the future of "lake house" technology.
  • Tiered storage in Kafka facilitates more efficient and cost-effective data management by decoupling storage from computing.

Podcast series: Data (R)Evolution

Topics around data, innovation and economics in a rapidly-changing cloud world.

Data drives innovation and revolutionizes the tech world. Data (R)evolution, powered by Aiven, explores the intersections of data, AI, innovation, and economics through the ever-evolving landscape of cloud technology. Led by hosts Francesco Tisiot and Janki Patel Westenberg, navigate this digital revolution in conversation with experts at the forefront of change.

Subscribe and never miss an episode

Watch the series

Hosts

Francesco Tisiot

Field CTO & Head of DevEx, Aiven

Francesco comes from Verona, Italy and works as a Field CTO & AI Lead at Aiven. With his many years of experience as a data engineer, he has stories to tell and advice for data-wranglers everywhere. Francesco loves sharing knowledge with others as a speaker and writer, and is on a mission to defend the world from bad Italian food!

Janki Patel Westenberg

AI, Platform, Open Source Product Marketing Leader, Aiven

Over the last 10 years, Janki has successfully led technical partnerships collaborating with Google Cloud and AWS to build database and caching solutions for gaming and media. Additionally, she has spearheaded GTM strategy partnered with AWS' product leadership for 19 AWS Analytics and Database services for multiple industries. Janki also is a proud alumna from University of Wisconsin-Madison (Go Badgers!)...fun fact, she studied Genetics and did research before moving to the cloud industry!

Guests

Josep Prat

Open Source Engineering Director, Aiven

Josep Prat is a Open Source Software Director at Aiven and is passionate about Open Source Technologies. With a strong background in Scala and distributed systems, he contributes to several Open Source Projects around streaming technologies. In his spare time, Josep enjoys late and long brunches on Sundays.

Filip Yonov

Director of Product, Data Streaming Services

Filip has many years of experience in consulting, IT project management, and product development. His previous positions include being responsible for the data streaming platform at ING with Apache Kafka and its ecosystems of 40 ING markets. He has experience as a senior product owner at REWE Digital and as a product owner at IBM for Docker Containers.