GoTo Financial goes far with Aiven

Overview

GoTo Financial needed a managed, region-specific Apache Kafka® cloud solution for a massive-throughput system, and they needed it quickly. They signed up with Aiven, and found a platform that just quietly works like it should, without any drama.

Highlights

• Just works without drama
• Massive throughput needs met
• Plugs into a huge ecosystem of apps
• Old fragmented Kafka architecture consolidated into a single system
• Easy orchestration with Kubernetes and Terraform
• Tiny in-house team performed the setup independently

The challenge

GoTo Financial is part of the Gojek ecosystem, a massive cluster of apps that combines e-commerce, on-demand and financial services to provide services and income opportunities for 270 million consumers and 11 million merchants in Southeast Asia. 

Massive load

The Gojek apps handle enormous throughput every second of every day. Millions of people depend on them for their next meal. Literally, because that’s where they order it, along with their rides and consumer goods and other things. But also in the sense that the apps enable people to earn their living by providing those services.

GoTo Financial’s GoPayLater app is integrated into all Gojek services as a payment method, and sees enormous traffic second to second. So it’s easy to see why lag and downtime must be kept to a minimum, and why information security and compliance with legal requirements are prime concerns.

Absolutely managed

From the outset it was clear to GoTo Financial that their Apache Kafka should be a managed service.

“We are a pretty lean engineering team,” says Kashyap Kopparam, Lead Engineer at GoTo Financial. “We didn't really have the capacity to manage our own Kafka and everything.”

The GoTo Financial Services tech team had zero desire to spend time and money on ramping up a new function to manage the sometimes arcane Kafka. The trouble was, they knew they had to choose a cloud service provider who had a data center within Indonesia in order to be legally compliant. In practical terms, that meant going to GCP and their Jakarta-based cloud region. But GoTo Financial was unable to find a first-party provider for Apache Kafka on GCP.

Of the options they explored (and there weren’t that many), Aiven offered an easy way to specify a region. They did their due diligence and were happy, even impressed with Aiven’s uptime and out-of-the-box compliances and certifications.

The extra mile, or fifty

Since they now had a big move on their hands anyway, GoTo Financial decided to re-architect their data pipelines from the ground up. Where they previously had separate Kafka instances for each product and function, they now wanted to congregate everything into a large consolidated Kafka system.

With Aiven, this was relatively risk-free where cost was concerned. Regardless of service size, Aiven is able to offer easy scalability both up and down. In the event of usage spikes in one service, GoTo Financial could firstly spread the load but also easily acquire more capacity, and they wouldn’t be stuck with the extra when the load evened out.

The solution

“It was definitely not a small thing to move from one cloud to the other, let alone another region,” says Kashyap. “And everything's being brought in from a different Kafka service. Besides, we’re a consumer facing product for pretty much everyone who has a mobile device in Indonesia, so the scale is also pretty big and we cannot afford downtime.”

Kashyap explains their migration process in brief: “We duplicated everything across both clouds, both the data and the systems. And then we moved over the user actions to the new region.”

Reliable, convenient and solid

GoTo Financial’s updated solution on Aiven for Apache Kafka offers great reliability and ease of use for their customers, both consumers and businesses. Because all the services are integrated into one system, the end user can stay up to date with their orders and with their wallet.

If a financial services customer with an outstanding loan makes partial repayment, they used to have to wait for the balance to update across services before being sure their payment went through. Now they instantly receive a push notification if the app transaction was successful. When the actual balance transaction is completed, the app shows the new outstanding balance. This makes a world of difference to the user experience.

And because no transaction is ever lost, even when connectivity is bad or there is a hardware failure, both the customer can be secure about their overall financial status--and GoTo Financial can be sure that payments don’t go astray.

“Gojek end users can use different financial services together with consumer services,” explains Raghunath SV, Product Manager at GoTo Financial. “For example, they can book a ride or buy groceries with prepaid credit, or they can take out a loan that allows them to pay for all purchases at the end of the month.”

The outcome

Half a year on, GoTo Financial is very happy with their new solution.

“We have much fewer services and less infrastructure to take care of now,” says Kashyap. “And all of it can be centrally managed.”

Because they’re using Kubernetes and Terraform, Aiven-related management tasks are trivial for them, and happen in the same environment as all their other infrastructure management.

Another benefit is centralized reporting. As a company, they get high-quality data for their business intelligence, allowing them to make more informed decisions about the directions they should steer their operations. Customer service also gets complete data on cross-pollinated products, which in turn helps them help the end users.

Kashyap also lists improved access control as a definite benefit. “We can give developers role-based access to Kafka, according to what they need to do there.”

Aiven enables the awesome

To emphasize it again, so no one misses it: this is their solution. The benefits they see are about 90% due to GoTo Financial’s team re-engineering their infrastructure from the ground up. From the get-go, Aiven’s role has been that of enabler. With Aiven, GoTo Financial’s new infra simply works like it’s supposed to. They haven’t even needed Aiven’s famed support.

“We just did it ourselves,” Kashyap shrugs. “We did everything ourselves, basically. I think there was one time when we asked Aiven to change a service name, but that's it.”

Cost control and predictability, though, is much improved with the move to Aiven, where the transparent pricing model allows GoTo Financial a panoramic view of their future infrastructure costs.

Their experience with Aiven as a service provider has been low-key but positive. “We haven’t seen any performance issues, we haven't seen any downtime. So yeah, we're happy with the system. That says a lot, because with other providers we’ve definitely experienced downtime,” Kashyap says. “Aiven is definitely one of the top reliable providers that we’ve dealt with.”

What’s next?

With GoTo Financial’s infra humming along nicely and steadily, Kashyap is starting to look into the future for added improvements. “There are nice product offerings around Aiven Kafka, like the Schema Registry, which we have not yet fully taken advantage of.” With Aiven, he’s sure of being heard. “Aiven gives off a very open and collaborative vibe. It’s easy to make suggestions and feature requests that would make our lives even easier.”