Data (R)evolution: Navigating Kafka: Challenges, solutions, and the future of real-time data streaming

Navigating Kafka: Challenges, solutions, and the future of real-time data streaming

In this Data (R)evolution episode, Matan Mizrahi and Filip Yonov join us to discuss how Kafka effectively handles and optimizes data streams, the pursuit of standardized APIs, and the vital role of AI in optimizing complex systems. They also share how AI and open-source tools are transforming the landscape. Tune in to hear about the evolution of data streaming and the transformative impact of AI and cloud-native solutions.

Key takeaways:

Emphasize the need for customer-defined scaling logic based on metrics and auto-scaling events for efficient Kafka management.

The importance of integrating AI, like the Ivan AI database optimizer, to enhance workload optimization and complex system management in Kafka.

The significance of strategic partner relationships in providing advanced software solutions and necessary knowledge and experience to mitigate risk in critical data infrastructure projects.

Podcast series: Data (R)Evolution

Topics around data, innovation and economics in a rapidly-changing cloud world.

Data drives innovation and revolutionizes the tech world. Data (R)evolution, powered by Aiven, explores the intersections of data, AI, innovation, and economics through the ever-evolving landscape of cloud technology. Led by hosts Francesco Tisiot and Janki Patel Westenberg, navigate this digital revolution in conversation with experts at the forefront of change.

Subscribe and never miss an episode

Watch the series

Podcast

Empower professionals to perform better with AI

Conversation about application of AI in database management, the importance of expertise in cloud adoption, and the need for streamlined tools in an increasingly data-driven world.

Jul 9, 2024

Watch podcast

Podcast

Hot and cold data with Apache Kafka, Tiered Storage, and Iceberg

Conversation about the critical importance of collaboration and feedback in enhancing Kafka's capabilities, the future of "lake house" technology.

Jul 16, 2024

Watch podcast

Podcast

The rise of multicloud, open source, and BYOC

Conversation about how to tackle the challenges of scaling a company, benefits of multicloud flexibility, the role of organizational sponsorship, and the innovative "Bring Your Own Cloud" (BYOC) concept.

Jul 23, 2024

Watch podcast

Hosts

Francesco Tisiot
Field CTO & AI Lead, Aiven
Francesco comes from Verona, Italy and works as a Field CTO & AI Lead at Aiven. With his many years of experience as a data engineer, he has stories to tell and advice for data-wranglers everywhere. Francesco loves sharing knowledge with others as a speaker and writer, and is on a mission to defend the world from bad Italian food!

Janki Patel Westenberg portrait — Janki Patel Westenberg
AI, Platform, Open Source Product Marketing Leader, Aiven
Over the last 10 years, Janki has successfully led technical partnerships collaborating with Google Cloud and AWS to build database and caching solutions for gaming and media. Additionally, she has spearheaded GTM strategy partnered with AWS' product leadership for 19 AWS Analytics and Database services for multiple industries. Janki also is a proud alumna from University of Wisconsin-Madison (Go Badgers!)...fun fact, she studied Genetics and did research before moving to the cloud industry!

Guests

Filip Yonov portrait — Filip Yonov
Director of Product, Data Streaming Services
Filip has many years of experience in consulting, IT project management, and product development. His previous positions include being responsible for the data streaming platform at ING with Apache Kafka and its ecosystems of 40 ING markets. He has experience as a senior product owner at REWE Digital and as a product owner at IBM for Docker Containers.

Matan Mizrahi portrait — Matan Mizrahi
Senior DevOps Engineer, Claroty
Matan is a talented Senior DevOps Engineer with extensive experience in cloud architecture and infrastructure. He enjoys experimenting with new technologies and introducing innovative solutions, always keeping dynamic demands and product needs in mind. Matan is passionate about enhancing the collaboration between R&D and DevOps, effectively bridging the gap between infrastructure and application development.