How Endowus is leveraging Kafka and microservices

As a financial technology company, Singapore-based Endowus has to abide by strict service standards and rules imposed by the country’s financial regulator, including allowing its customers to redeem their mutual funds at all times.

That means the startup, which has more than S$2bn of assets under management, cannot afford to build its investment platform as a single application as that could take down the whole service if any part of the application fails.

“Separating the application into multiple applications and running them in physically isolated servers is a requirement and not an option for us,” said Joo Won Lee, chief technology officer and partner at Endowus.

To ensure its platform is resilient and can recover quickly from outages, among other requirements, Endowus adopted a distributed microservices architecture where services communicate with each other through Apache Kafka, the open-source distributed event streaming platform.

“We have a redemption service that just handles redemption requests, and that service must be live all the time,” said Lee, adding that the Endowus platform also interacts with other services such as SingPass, Singapore’s national identity system, for customer onboarding.

Through a managed Kafka service from Confluent, Endowus has been collecting event streaming data on customer interactions and other insights. This could range from drop-off points during onboarding to variations in risk appetites of customers.

Endowus has already acted on the onboarding data, which is now being used by a team of engineers and product managers to reduce onboarding time and improve customer conversion rates. “What we could potentially do further is A/B testing, which we are not doing enough of in our digital marketing campaigns,” said Lee, adding that this could include testing which landing pages resonate better with customers, for instance.

But being a nimble, born-digital company that works with microservices has not been easy for Endowus. As it focuses on managing investment portfolios and providing advisory services, it has to rely on UOB Kay Hian, a local brokerage firm, to hold customer deposits and reconcile transactions for its customers in Singapore.

“Not all partners we interact with are as digitally savvy as us,” said Lee, adding that UOB Kay Hian, for example, provides data in the form of files rather than events, which means Endowus has had to convert the data in those files into events in a standard format.

Endowus did not always rely on a managed Kafka service. It only signed up for Confluent just over two years ago, when it was growing rapidly and decided not to manage its own Kafka infrastructure.

“Engineers are expensive, and we probably won’t be able to hire many of them,” said Lee. “Plus, our investment team is consistently coming up with new product ideas which we want to support, rather than spend time managing Kafka clusters and worrying about security patches.”