Reliable, scalable and compliant data infrastructure supports Dojo’s experience economy across Europe
Aiven makes setting up cloud databases so simple anyone can do it. Our set-it-and-forget-it solutions take the pain out of cloud data infrastructure.
Dojo, a rapidly scaling UK tech company, is expanding its operations across Europe and adding new data and AI-driven services to its payment processing services for businesses. With Aiven, the organization has implemented a new data streaming platform that integrates multiple cloud environments, databases and tools. This provides the speed, scalability and resilience that Dojo’s business requires — while giving its teams the autonomy to innovate with data and better serve its customers.
Dojo is one of the fastest growing tech companies in Europe. It specializes in payment processing services, handling £2.2 million in UK-based transactions every half hour — and more than 35 million transactions every week.
Today Dojo has more than 150,000 customers and a team of more than 1,000 employees with several offices in UK alone. Initially a card-payment provider for small businesses, it has expanded its services to include Dojo Bookings and the Dojo restaurant finder application, formerly WalkUp, popular with 600 restaurants and 600,000 diners. Dojo is expanding its European footprint and has launched in Southern Ireland and Spain.
“In a heavily regulated industry like payment processing, data reliability is essential. It demands strict guarantees around the security and speed of all payment data,” says Jérémy Barneron, Senior Software Engineer, Dojo. “It is not acceptable for Dojo to lose a single transaction, miss a settlement or experience a delay in processing as this would compromise our customers’ cash flow and revenue streams.”
Dojo prioritized speed and reliability from inception, building a multi-cloud payment solution on AWS, Google Cloud and Oracle Cloud. This microservices-based architecture enabled rapid feature releases and market leadership while mitigating vendor lock-in and enhancing resiliency.
This flexible approach enabled the various teams at Dojo to choose a combination of cloud services and databases that worked best for their use cases. However, Dojo found it challenging to navigate the increasing complexity of managing, and extracting value from, data generated by multiple sources.
This complexity was also hindering the company’s plans to expand both its services and geographical footprint. “With our ambition to scale up, we had a hard requirement to streamline communication throughout all of the services and all the different functions of the business,” says Jérémy.
At the same time, Dojo understood the importance of data quality and governance. It relied on tooling from each of the cloud providers, many of which fell short of Dojo’s requirements. As a result, Dojo’s engineers were responsible for taking time-consuming, additional measures on each cloud platform to ensure the tooling remained secure and compliant with regulations like GDPR and PCI.
To overcome these challenges, and to increase its data analytics capabilities, Dojo wanted to create a centralized data streaming and messaging architecture. It decided to publish all its data to Apache Kafka®, an open-source distributed event streaming platform that can handle high-volume, high-velocity and high-variety data streams at very low latencies.
But Dojo also wanted a managed Kafka service, and one that was compatible with multiple clouds across a variety of geographical regions. After evaluating the market, it chose Aiven for Apache Kafka. “We work with lots of cloud providers, so we need a solution that fits with them all. We also wanted an open-source solution as we try to avoid vendor lock-in. Aiven was the perfect match,” says Elad Leev, Senior Data Platform Engineer, Dojo.
“Within a year, we built a complete end-to-end data streaming platform that met all our demands for reliability, scalability and fault-tolerance, and which could deal with any data challenge. Aiven for Apache Kafka is at the heart of this environment.” says Jérémy.
All the data generated from different systems, for example from settlement, clearing and billing, still runs on different clouds and databases such as OracleDB, MongoDB, PostgreSQL and Google Spanner. But now the majority of the data is published to Aiven for Apache Kafka running on Google Cloud, which is then synched according to the unique needs of various teams and functions of the business. For example, a significant volume of data goes to Google BigQuery for data analytics and to feed Dojo’s various AI and ML systems.
“With the Aiven Platform, our teams have the autonomy to select the database and cloud solution they believe will work best, and to shape the data to solve their specific use case,” Jérémy says.
For greater resilience, Dojo has adopted a cross cloud disaster recovery strategy, using several regional sites that replicate the data to other cloud providers in real time.
With the new data streaming platform in place, Dojo is seeing improved data quality and data governance. The open-source streaming ecosystem and tools around Apache Kafka provides Dojo with critical capabilities like Change Data Capture (CDC) through Aiven for Apache Kafka® Connect.
This technology continuously captures changes in data as they occur and propagates the changes in real time to Apache Kafka to keep all downstream applications in sync. CDC is a linchpin in Dojo's commitment to robust data ingestion, governance and compliance. Jérémy says, "It’s really valuable to us that Aiven provides managed open-source tooling around Kafka, such as Kafka Connect. It helps us guarantee the right levels of consistency in our systems.”
“It was a big selling point that Aiven provides a fully open-source schema registry for managing data right out of the box — in fact it was a game-changer,” says Jérémy. “It reduces our technical debt, avoids proprietary solutions and allows us to focus more on innovation and what matters for our customers.”
The real-time data streaming capabilities of Kafka enables Dojo to perform essential business functions much faster. For example, as a non-bank payment service provider, Dojo is legally obliged to protect or ‘safeguard’ customer funds as a transaction comes in from its payment systems. Safeguarding requires checks and API calls to different providers and security companies, which can be completed significantly faster through their event-driven architecture powered by Aiven for Apache Kafka.
The new architecture with Kafka at the center enables Dojo to add, evolve and scale microservices components independently, providing agility and flexibility as volatile markets and customer trends change. “We have unprecedented levels of agility now which helps us enormously as we expand, set up new sites and launch new products,” says Jérémy.
In addition, with Kafka in place, Dojo has improved its AI and data analytics capabilities, which helps both the company and its customers. For example, Dojo’s business-funding service relies on real-time data from Kafka and Dojo’s AI model to assess risk and provide Dojo's customers the right business funding to help them grow.
Streaming data from Kafka also helps Dojo’s ML and AI models anticipate customer churn so Dojo can take immediate proactive steps to retain that customer. “Data loses its value as time goes by,” says Elad. “Now, with Aiven for Apache Kafka, we can react to business events very quickly.”
And on the customer side, one of Dojo’s central missions is to revolutionize the “experience economy” by empowering customers with data-driven products and services. With the new data infrastructure in place, Dojo uses data analytics to give customers valuable insights about their own operations and to enable them to make intelligent business decisions.
Dojo’s teams now have the autonomy to keep designing services that make life better for customers across Europe. Jérémy’s team is currently exploring stream processing use cases and how Aiven might help with more complex, real-time operations on top of streams. Jérémy concludes, “We're planning our future with Aiven — we really want to push our collaboration forward.”
