Alef reaches over 120,000 students with Aiven

Overview

Alef Education, founded in 2016, is a leading K-12 education technology company, making learning more personalized, engaging, and accessible for students. Its Alef Platform allows students to complete exercises online and receive instant feedback from teachers. The AI-powered platform also provides real-time visibility for teachers, informing them where students are struggling — saving time while facilitating valuable teaching opportunities by making teachers more reactive to students’ needs. The platform is used today by more than 120,000 students in over 400 schools in the United Arab Emirates, the United States of America, and Canada.

Running this type of platform at scale requires processing and managing a tremendous amount of real-time data. When the company decided to scale up their operations back in 2017, they were faced with a daunting task: how to do this without compromising reliability and performance even temporarily.

The challenge

Alef Education’s learning and big data platforms are built around the use of event-driven microservices, and this type of architecture is difficult to scale. Also, as data cannot transfer across multiple microservices, processing data and ensuring a seamless high-quality user experience is demanding.

What’s more, the company was experiencing throttling issues with messaging broker RabbitMQ. This led to significant performance issues — to the point where students were having difficulty receiving real-time feedback on their work.

Something had to be done to improve performance and enable scalability, so Alef Education migrated to Apache Kafka, an open-source distributed event streaming platform. The company opted to run Kafka on AWS and manage it using in- house resources, which proved to be quite a difficult undertaking. They already had a wide variety of technologies in use, but they discovered that they lacked the specialized resources to manage Kafka on their own.

The solution

During this time, Alef Education had about 25 in-house engineers. Given Kafka’s complexity and the difficulty to operate it, Alef Education found that they could not afford to support the entire technical spread they needed, with Kafka being the major pain point.

“I realized that running Kafka in-house would take up valuable time from our team to work on more pressing matters,” explains Muhammad Noor, Head of Engineering at Alef Education. “So we decided to look for other options.”

Noor and his team set out to find a managed cloud services provider to improve platform performance and discovered Aiven. The team was immediately impressed by Aiven’s responsiveness, knowledge, and service maturity.

Initially Alef Education used Aiven just for big data storage, but after experiencing Aiven’s industry-leading quality of service and customer support, the company decided to migrate their entire data processing core to Apache Kafka. Aiven now serves as the main pathway for data transfers between their student learning platform and big data analytics engine.

The outcome

Since implementing Aiven for Kafka, Alef Education has drastically improved its scalability and performance, and is now in an excellent position to handle increased workloads as the demand for remote learning grows.

According to Noor, Aiven for Kafka has saved a tremendous amount of work for the team at Alef Education. This has en- abled them to focus more attention on more important tasks: “The team is more focused now on delivering business value than being bogged down with how to run different services like Kafka. That burden has been eliminated and we are more concerned about delivering to our customers instead of being bothered by ticketing issues.”

In fact, Alef Education was so pleased with Aiven for Kafka that they also rolled over their entire Grafana monitoring to Aiven for Grafana, which they use in combination with Aiven for InfluxDB. Moreover, Alef Education has drastically reduced latency in their learning application, using the combination of Aiven for Kafka and an Aiven Virtual Private Cloud (VPC) — resulting in faster feedback for learners. In the past, students had to deal with three seconds of latency when waiting for feedback. Aiven has cut that latency in half.

In addition, the Alef Platform is now more reliable. By using a managed service, Alef Education is able to set a concrete uptime target for its platform, and Aiven has reliably met their 99.99% uptime guarantee. This allows the company to com- fortably serve over 400 schools without having to worry about unexpected outages. This is a major differentiator, especially considering the fact that many online learning platforms are prone to outages during periods of heavy load. In contrast, the schools, teachers, and students using the Alef Platform can rely on its security and reliability.

As an added bonus, Aiven has reduced their overall develop- ment costs. Previously, Alef Education’s engineers spent about 20 percent of their time managing Kafka. Thanks to Aiven, they no longer have to spend any time on it, giving them more hours to focus on their core business.

“Onboarding Aiven was a more cost-efficient solution for Alef Education than hiring developers in-house — and for a more reliable, more scalable service,” explains Humayun Jamal, a cloud/DevOps architect at Alef Education.

Thanks to Aiven, the Alef Platform can scale now without limita- tion, and with push-button ease. Alef Education has grown from serving 10 schools to over 400 and now handles over 2 million answered questions per day — seamlessly and without fear of system errors and user complaints, with the support of Aiven.

“Thanks to the reliability that Aiven provides, we have more confidence in our ability to scale up and serve larger clients,” Noor added. “I think that’s what people are looking for from any managed service provider — confidence and comfort,” Noor concludes. “I would recommend Aiven to any business that’s looking for a managed solution provider”