Writing highly scalable backends in UDP
I'm Glenn Fiedler and welcome to Más Bandwidth, my new blog at the intersection of game network programming and scalable backend engineering.
Imagine you have a system you need to code and it needs to scale up to many millions of requests per-second. The well-trodden path is to implement this in HTTPS with a load balancer in front of some VMs that autoscale according to demand. For example, you could implement this in Google Could with load balancer in front of a MIG and implement the HTTPS handlers in Golang (trivial). There's plenty of information about how to do this online, and it's relatively easy to scale this up to many millions of requests per-second.
But what if you needed to do the same thing in UDP instead of HTTPS?
Well, now you're off the beaten path, and you'll find very little information about how to do this. This is actually something we needed to do at Network Next, my startup that provides network acceleration technology for multiplayer games. Our SDK runs on game consoles like PS4, PS5, Nintendo Switch and XBox as well as Windows, MacOS and Linux, and avoiding the overhead and complexity of porting and maintaining something like libcurl or mbedtls on consoles is beneficial to our customers.
To be clear, I'm not advocating that you stop using HTTPS and switch your backend to UDP. If you're happy with HTTPS and it's doing what you need, awesome! Stay on the well trodden path. But if, like me, you have some use case that is better with UDP, or even if you are just curious about how such a strange approach can work, read on.
The Perfect Interview Question
This idea of building a scalable backend with UDP is so out there that I've used it as an interview question at Network Next for years. You simply cannot just google this and find example source code showing how to do it. To solve this problem, you need to take in many sources and creatively synthesize your own result. Exactly what I'm looking for from engineers at Network Next.
The actual question itself is deceptively easy:
You are tasked with creating a client/server application in Golang that runs in Google Cloud. The client in this application must communicate with the server over UDP.
Each client sends 100 requests per-second. Each request is 100 bytes long. The server processes each request it receives and forwards it over HTTP to the backend.
The backend processes the request, and returns a response containing the FNV1a 64 bit hash of the request data. The server returns the response it receives from the backend down to the client over UDP.
Implement the client, server and backend in Golang. Provide an estimate of the cost to run the solution each month at a steady load of 1M clients, as well as some options you recommend as next steps to reduce the cost.
While I'm confident that an experienced senior engineer could find a solution over a weekend, I gave engineers as much time to research and solve the problem at home as they needed. What matters is the thinking process of the engineer, not how quickly they implement it. And of course, I wanted to respect that engineers may be implementing this in their spare time while working another job.
If you'd like to have a go at solving this yourself, now is the time.
I'll publish the full solution April 16, 2024, one week from today.
