Read a roundup of this mecca for data scientists and big data enthusiasts, and get the latest on AI and ML trends in the Python community!
Celeste Horgan
|RSS FeedDeveloper Educator at Aiven
Subscribe for the latest news and insights on open source, Aiven offerings, and more.
Last week the German Python community hosted PyCon DE + PyData DE, for the first time in Darmstadt, Germany. Aiven was there in the form of myself, Celeste Horgan, Developer Educator at Aiven. It was my first time at a Python conference and I was pleasantly surprised by the warmth of the community and breadth of applications for the Python programming language.
Darmstadt and the surrounding region are host to some of Germany’s most prestigious universities and associated research institutions, most notably a branch of the European Space Agency. The conference had a particularly academic and science-focused flavor as a result. Many attendees were current or former physicists, data scientists and machine learning engineers, and the conversations on the conference floor reflected that.
This is perhaps best represented by the day 2 keynote by Dr. Guadalupe Canas-Herrera, Theoretical Cosmologist with the European Space Agency and one of the major scientists involved in the launch of the Euclid deep space telescope. Dr. Canas-Herrera took an impressed audience through the mission to map and understand the universe’s dark matter and dark energy using a camera multiple times more powerful than Hubble or JWST. Then, Dr. Canas-Herrera covered the open source data projects created to support the effort.
Notably, given the scale of the data generated by the Euclid telescope over its lifetime, Dr. Canas-Herrera emphasized both the challenges of managing that quantity of data and the impossibility of processing the data without the use of deep learning and machine learning algorithms. While AI/ML use is nascent in the broader industry, the Python community, with its deep scientific and data science ties, treats AI/ML as a given, not an exploratory topic.
The Eculid project has produced a number of interesting open source data artifacts, including the Euclid quick data release, a freely available dataset from the Euclid telescope’s most recent scans, and many interesting code artifacts available via the Euclid Consortium's ELSA project. Open data sets and the tools required to parse them are an important part of any community – Aiven uses many similar data sets in its demos and documentation!
But most of us – and certainly most of Aiven’s customers – aren’t gazing into the depths of the universe or generating petabytes of data in the process. Luckily, PyConDE had plenty of talks geared towards more earthly concerns!
In particular, I enjoyed Challenges and Lessons Learned While Building a Real-Time Lakehouse using Apache Iceberg and Kafka delivered by Elena Ouro Paz from the Schwarz IT group, the company behind popular grocery store brands like Lidl and Kaufland, and
Jonas Böer from their implementation partner inovex.
This talk was a case study of using Apache Kafka® and Apache Iceberg™ as intermediaries between an Oracle-based ERP system and Snowflake® as a service layer for analytics data. In implementing Kafka, Schwarz IT was able to achieve near real-time display of analytics data - a much improved situation from a system which previously took many minutes to render a view for its users. While the talk focused on the Iceberg implementation, the team did some clever analysis of their data and optimized their Kafka accordingly. Choosing Apache Kafka also allowed the team to make use of a schema registry to handle data drift between the dozens of data sources feeding into this system.
One of the major points Elena and Jonas highlighted was the reason for moving away from a pure Snowflake solution was the astronomical cost of compute and data storage. Lidl operates in 35+ countries with hundreds of stores and millions of orders processed per day – in addition to a desire to render analytics data in realtime, Elena noted that there was a desire to implement as much open source technology as possible to help keep costs low.
The vast majority of talks on the schedule were related to AI, ML, deep learning, AI Agents and LLMs. But at the Pyladies lunch on Thursday, one question seemed to circle around the table: were we really doing anything new?
From the perspective of the Python data community, nearly all LLM use cases are identical or extensions of classical deep learning and machine learning use cases, with the addition of (occasionally unpredictable) large language models. Indeed, attending some of these talks supported this feeling. That said, there was one category of AI/ML talks that emerged as an exciting new idea at PyConDE: Model context protocol.
The most exciting part of the AI/ML talks at PyConDE were definitely those on Model Context Protocol – these led to multiple talks on building practical business AI applications. Notably, BlueYonder’s Martin Seeler went through how they built an advanced AI agent to handle and coordinate the back-end systems which drive most e- and regular commerce applications. E-commerce is one of the most complex use cases to develop for, given the number of domain-specific tooling and interconnected systems at play.
Similarly, Rodel van Rooijen gave a very hands-on talk about building an analytics-generating chatbot using the same techniques.
Strikingly, most of the talks on Model Context Protocol at this event were focused on actual use cases, in contrast with the more theoretical approach we saw last year, both at events and in conversation with customers. This suggests that thoughts around AI and LLMs are shifting from theory and hype into practical implementations, with Model Context Protocol becoming an active bridge into practical AI applications.
Model Context Protocol (MCP) is an open standard from Anthropic which allows businesses like Aiven to expose its services in a way that large language models and chatbots can interface with. This allows for agentic AI development, or, creating chatbots which can take actions on behalf of a user. For example, a customer service agentic AI could record an interaction they had with a known customer in a database through using the tools provided by an MCP server.
This was a hot topic of discussion among Python developers at PyConDE, and we expect that interest in developing with MCPs will continue to grow into the next year. They’re a powerful way to create chatbots for complex tasks both internal to your business and external to customers.
If you’re interested in working with your Aiven data stores using large language models, we’ve just released our Aiven Model Context Protocol server for your consumption!
The highlight of the lightning talks was definitely Lisa Quatmann’s "Recruiters Hate this One Weird Trick [investing in the community for fun and profit]” where she builds an entire engineering team on less than 5000€ by sponsoring Python community events like Django girls and the Python user group in Berlin.
In difficult economic times, companies often start questioning their investment in open source and in community events, but Lisa’s talk shows that it has tangible financial benefits to a company. PyConDE also sponsored spaces for PyLadies, DjangoGirls, and the very cool Feminist AI LAN Party.
Again – spaces like these can go underappreciated in conferences, but the sheer number of women and minority attendees at PyConDE, and the proportion of those women in “serious” technical roles like data science shows that these community outreach programs work and bring talent into the field.
One of the Aiven DevReal team's biggest commitments in the coming year is to support the open source communities and projects we use to build our business, and attending PyConDE is a part of that. In the coming months, you can find our team elsewhere, too – we’d love for you to say hi!
We'd love to say hello to you, so look out for us!
