Alef's digital learning platform runs on Apache Kafka, bringing content to learners and insights to teachers.
Aiven makes setting up cloud databases so simple anyone can do it. Our set-it-and-forget-it solutions take the pain out of cloud data infrastructure.
Founded in 2016, Alef Education is a leading K-12 educational technology company that makes learning more personalized, engaging, and accessible for students. Its flagship product, the Alef Platform, allows students to complete exercises online and receive instant teacher feedback. The AI-powered platform also provides real-time visibility for teachers, informing them where students need additional support. It saves time and makes teaching easier as teachers can better respond to students’ needs. The platform is used by more than 1 million students in over 14,000 schools in the United Arab Emirates, the United States, Indonesia, and Morocco.
Operating this platform at scale requires the processing and managing of an enormous amount of real-time data. When the company decided to scale its operations in 2017, it faced a daunting task: how to do so without compromising reliability and performance, even if only temporarily.
The learning and big data platforms at Alef Education are built around event-driven microservices. This architecture requires more effort to scale. Since data cannot be transferred across multiple microservices, processing data and ensuring a seamless, high-quality user experience is challenging.
In addition, the company was experiencing throttling issues with the messaging broker RabbitMQ, resulting in significant performance issues that impacted real-time feedback for students on their work. To improve performance and enable scalability, Alef Education migrated to Apache Kafka, an open-source platform for distributed event streaming. The company chose to run Kafka on AWS and manage it with internal resources, which proved to be a complex undertaking. They already had various technologies in place but needed more specialized resources to manage Kafka independently.
During this time, Alef Education employed around 25 engineers. Given the complexity of Kafka and the difficulty running it, Alef Education found that they needed help to support the full range of engineering required, with Kafka being the critical pain point. “I realized that running Kafka in-house would take away valuable time from our team that we could be spending on more pressing tasks,” explains Muhammad Noor, Head of Engineering at Alef Education. “So we decided to look for other options.”
Noor and his team set out to find a managed cloud services provider to improve the platform’s performance and came across Aiven. The team was immediately impressed by Aiven’s responsiveness, knowledge, and service maturity.
Initially, Alef Education used Aiven for big data storage. After experiencing Aiven’s industry-leading quality of service and customer support, the company decided to migrate its entire data processing core to Apache Kafka. Aiven is now the primary way to transfer data between the student learning platform and the big data analytics engine.
Since implementing Aiven for Kafka, Alef Education has dramatically improved its scalability and performance. It is now in an excellent position to manage increasing workloads as demand for distance learning increases. According to Noor, Aiven for Kafka has saved the Alef Education team a tremendous amount of work, allowing them to focus on more important tasks: “The team is now more focused on delivering business value rather than struggling with running various services like Kafka.” That burden has been removed, and we are spending more time delivering excellent customer service instead of dealing with ticket issues.”
Alef Education was so happy with Aiven for Kafka that they also moved all of their Grafana monitoring to Aiven for Grafana. Additionally, Alef Education has dramatically reduced latency in its learning application by combining Aiven for Kafka and an Aiven Virtual Private Cloud (VPC), resulting in faster learner feedback. In the past, students had to expect a latency of three seconds when waiting for feedback. Aiven halved that.
The Alef Platform has increased its reliability. By using a managed service, Alef Education can set a concrete uptime target for its platform, with Aiven reliably meeting its 99.99% uptime guarantee. This allows the company to conveniently serve over 14,000 schools without worrying about unexpected outages. It is a key differentiator, especially because many online learning platforms are prone to outages during busy times. In contrast, schools, teachers, and students using the Alef Platform can be confident in its security and reliability.
As a bonus, Aiven has reduced its overall development costs. Previously, Alef Education’s engineers spent approximately 20 percent of their time managing Kafka. Thanks to Aiven, they no longer have to spend time on this and have more time to focus on their core business. “Onboarding Aiven was a more cost-effective solution for Alef Education than hiring our own developers - and for a more reliable, scalable service,” explains Humayun Jamal, a Cloud/DevOps Architect at Alef Education.
Thanks to Aiven, the Alef Platform can now scale indefinitely and at the touch of a button. Alef Education has grown from ten schools to over 14,000 and currently processes over 4 million daily questions seamlessly and without fear of system errors and user complaints, with the support of Aiven.
“Thanks to the reliability that Aiven provides, we have more confidence in our ability to scale and serve larger customers,” adds Noor. “I think that’s what people want from a managed service provider — confidence and comfort,” concludes Noor. “I would recommend Aiven to any organization looking for a managed solutions provider.”
