Building out the Payments Platform

Hi, I’m Patrick, a senior software engineer in the Payments team.

As you may have already guessed, we’re responsible for any money flowing in and out of your Freetrade account. 

Today I’m going to give you a sneak peek under the hood of the Payments team and what we’ve been up to. 

In 2021, the Freetrade team increased from 89 people in the UK to over 300 people across the UK, Sweden, Australia and Canada.

This rapid growth meant that we had to split larger teams into smaller ones, each focused on a specific client area. In order to scale our engineering team, we had to also break up our codebase into smaller chunks.

Breaking Down the Monolith

If your eyebrows have raised at the word Monolith, check out this blog first. I’m going to delve into a slightly more technical perspective.

It’s no secret that we picked Google Cloud as our cloud service provider. 

One feature I really like about Google Cloud is the concept of projects. Projects are essentially logical groupings of cloud resources that, by default, are completely isolated from each other. 

So if you wanted to create a dev, test, and production environment, all you need to do is create three projects. In contrast, with AWS you would have to create three separate AWS accounts, with different emails and passwords.  

At the beginning of 2021, as a backend engineer, you primarily had to worry about two particular platforms (Google Projects):

1. The Client Platform, which deals with users, accounts and anything that works with our mobile apps.
2. The Invest Platform, which is our propriety brokerage platform.  

Initially, these two platforms were easy to manage when we were a handful of engineers. As we grew, however, we started to see pain points in code ownership, shared releases, and maintainability. 

As a newly formed Payments team, we took the opportunity to branch out our new features into a new Google project.

Having our own Payments Platform comes with plenty of advantages. As a team, we would have better control over architectural decisions, release processes and strict feature segregation. 

Additionally, Google Cloud quotas are usually defined per project, therefore splitting up into separate projects would improve Freetrade’s ability to scale with its rapid user growth.

Finally, billing information can be viewed at a project level - crucial for making costing decisions and keeping operational costs low. 

Naturally, given a new clean slate to work on, we set out to find the best tools to build our new payments platform.

Moving away from Serverless Functions to Serverless Containers

Freetrade has been reliant on Serverless Cloud Functions for a long time. They have allowed us to scale our resources at a time when user numbers have been growing rapidly. 

However, Cloud Functions aren’t perfect. 

You have to deal with cold starts, short life spans, and most of all, a lacking locally testable solution during development (out of the box, you can’t just spin up a function and test it out). 

This final point encouraged us to look into what Google Cloud could offer around containerisation. This wouldn’t be the first time that Freetrade would look into using something like Kubernetes - Tom Waszkowycz explained how live pricing was implemented using GKE Autopilot. 

The Payments team, however, went with Google Cloud Run as their go-to cloud service.

‍

‍

If GKE and Google Cloud Functions had a baby, they would have named it Google Cloud Run, because it is essentially serverless containers. 

This means it comes with all the benefits of serverless (cheap, scalable, effortless), can have a maximum timeout of 60 minutes, and just like with Kubernetes, you can set the minimum number of active instances, and keep the CPU warm for those handy long-running reconciliation processes.

With regards to programming languages, we felt that it was best to stick to Typescript as it was already used widely at Freetrade. We also use Terraform for configuring our infrastructure. 

During development, while you can test our services locally using Docker compose, we also have a shared dev environment that we can easily deploy to using Terraform.   

Which Firestore?

Our choice of database is Datastore. 

Interestingly, the original Datastore service was deprecated and relaunched as Firestore in Datastore mode. Confusingly, this option is quite obscure as in the Firebase portal you can only see Firestore in Native mode, which is the Firestore Freetrade has been using all along in its other projects. 

But wait, there's more. You can only pick one of these modes per Google project and once picked, there's no going back! The only way to switch is by recreating a new project and starting again. 

‍

‍

Nonetheless, there are some advantages and disadvantages of picking Datastore Mode over Native Mode. 

Of the two, Datastore mode boasts higher performance, being able to scale to millions of writes per second. It also provides an SQL like querying language called GQL. For the most part, Datastore is also cheaper since small operations are free. 

Datastore's downsides include the fact it’s missing the real-time features that Firestore in native mode offers like database triggers and streams and, to be frank, the UI for native mode is far better than Datastore. 

In the end, we decided that for Payments, having a higher throughput performance while keeping costs low was more important than having the real-time features, that are mainly aimed toward mobile development. 

In principle, we also try to use event sourcing when it makes sense to do so. 

Now that we have a dedicated team and platform for Payments, we can focus our efforts on building out new payment features and integrations as we expand across Europe and beyond.   

‍

Where you come in

We are rapidly expanding the payments team and are constantly on the lookout for Senior Software Engineers. 

We’d love to have you join our team! Check out our roles available!

‍

‍