Scaling a Node.js API With Google App Engine

A few months ago I was faced with an issue I’ve never had to deal with. A Node API that I built needed to scale from 10,000 requests per month to 130 million requests per month. Before it was just a layer on top of our legacy PHP code to serve 3rd parties, but then we migrated all of the functionality from PHP into the Node API. Hence the huge change.

Using a great tool called Loader.io by SendGrid, I found out that placing 50 requests per second on the API would result in a 7,400ms response time on our $40/mo. DigitalOcean server. Seven seconds for an API call just wouldn’t work. It needed to be in the 100 to 300ms range.

Another issue was that it was HARD to manage deployment. There isn’t an easy straight-forward way to manage deployment of a Node API on a VPS. And this API became the core of our application stack, so we needed a rock solid deployment plan as well.

Google App Engine solved all of those issues. 130 million requests per month is a piece of cake on App Engine. After doing some initial testing, we found that the average response time was well below my requirement of 300ms at 50/sec.

Here are my thoughts on GAE:

Pro: Deployment

App Engine makes it incredibly easy to manage versions of your application in the web console. You can stage and even split traffic between versions for testing. And deploying a new version is just a simple command with their gcloud utility. This is far far easier and more streamlined than our old deployment strategy which was often faulty and unreliable.

Pro: Auto-scaling

Since App Engine packages your code up with containers, your code doesn’t rely on the underlying file system or operating system. Unlike our old strategy with DigitalOcean, App Engine can automatically spin up a dozen instances of our API to handle any load size. Then destroy them as the demand decreases. All we have to worry about it paying the bill (which is cheaper than me fumbling with system administration several hours a month).

Pro: Speed

It’s Google, so… no problem here. 300ms? Hah, how about 30ms!

Pro: Integrations

Google Cloud Platform is finally at the stage where you can manage almost all of your application on it. We are using Cloud SQL for our relational data, Cloud Datastore for our non-relational data, Memcache for caching and extremely high speed data retrieval, Cloud Storage for file storage, GAE for our applications and services, Cloud CDN for serving our AngularJS applications, and many more GCP services. Bring your code, and they’ll take care of the rest.

Pro: Price

I’m a software developer, not a sys-admin. Not to mention that we have a very small team of coders. So we literally don’t have the time or resources to maintain infrastructure. GCP will cost 50 to 100% more than our old infrastructure, however, it will take the burden of sys admin work off of us poor coders. Plus, when doing the math, the extra cost is less than what the CEO pays me every month to maintain infrastructure (albeit pretty poorly).

Con: Learning curve

GCP and GAE learning curve isn’t much, but it does exist. I spent about 60 hours reading about and testing GCP to prepare for the migration, not to mention that I also had to train my staff on it before we got started. It was definitely a departure from our old way of thinking.


Overall Google App Engine (and the rest of Google Cloud Platform) has been a great choice for us. We are very happy with our decision, and genuinely enjoy working with GAE. We’ve drastically reduced our infrastructure maintenance time, and we know that GCP will be there for us when we need to scale even more.

Recommended reading: Installing a Specific Version of Node