Navigating Kafka: Challenges, solutions, and the future of real-time data streaming

In this Data (R)evolution episode, Matan Mizrahi and Filip Yonov join us to discuss how Kafka effectively handles and optimizes data streams, the pursuit of standardized APIs, and the vital role of AI in optimizing complex systems. They also share how AI and open-source tools are transforming the landscape. Tune in to hear about the evolution of data streaming and the transformative impact of AI and cloud-native solutions.

Key takeaways:

  1. Emphasize the need for customer-defined scaling logic based on metrics and auto-scaling events for efficient Kafka management.
  2. The importance of integrating AI, like the Ivan AI database optimizer, to enhance workload optimization and complex system management in Kafka.
  3. The significance of strategic partner relationships in providing advanced software solutions and necessary knowledge and experience to mitigate risk in critical data infrastructure projects.

Youtube thumbnail
logo-data-revolution.png

Podcast series: Data (R)Evolution

Topics around data, innovation and economics in a rapidly-changing cloud world.

Data drives innovation and revolutionizes the tech world. Data (R)evolution, powered by Aiven, explores the intersections of data, AI, innovation, and economics through the ever-evolving landscape of cloud technology. Led by hosts Francesco Tisiot and Janki Patel Westenberg, navigate this digital revolution in conversation with experts at the forefront of change.

Subscribe and never miss an episode

Watch the series

Hosts

Francesco Tisiot
Francesco Tisiot

Field CTO & AI Lead, Aiven

Francesco comes from Verona, Italy and works as a Field CTO & AI Lead at Aiven. With his many years of experience as a data engineer, he has stories to tell and advice for data-wranglers everywhere. Francesco loves sharing knowledge with others as a speaker and writer, and is on a mission to defend the world from bad Italian food!

Janki Patel Westenberg portrait
Janki Patel Westenberg

AI, Platform, Open Source Product Marketing Leader, Aiven

Over the last 10 years, Janki has successfully led technical partnerships collaborating with Google Cloud and AWS to build database and caching solutions for gaming and media. Additionally, she has spearheaded GTM strategy partnered with AWS' product leadership for 19 AWS Analytics and Database services for multiple industries. Janki also is a proud alumna from University of Wisconsin-Madison (Go Badgers!)...fun fact, she studied Genetics and did research before moving to the cloud industry!

Guests

Filip Yonov portrait
Filip Yonov

Director of Product, Data Streaming Services

Filip has many years of experience in consulting, IT project management, and product development. His previous positions include being responsible for the data streaming platform at ING with Apache Kafka and its ecosystems of 40 ING markets. He has experience as a senior product owner at REWE Digital and as a product owner at IBM for Docker Containers.

Matan Mizrahi portrait
Matan Mizrahi

Senior DevOps Engineer, Claroty

Matan is a talented Senior DevOps Engineer with extensive experience in cloud architecture and infrastructure. He enjoys experimenting with new technologies and introducing innovative solutions, always keeping dynamic demands and product needs in mind. Matan is passionate about enhancing the collaboration between R&D and DevOps, effectively bridging the gap between infrastructure and application development.