The Hacker News Effect - Handling Web Traffic Surges

We have found ourselves at the top of Hacker News a few times. It’s impossible to know what will trend but every so often send it to the top. It’s nice to have 15 minutes (or 24 hours) of Internet fame, and it also gives your web servers a good workout - a nice real-life performance test. We can watch the requests pour in, and see our infrastructure scaling up in real-time.

As the traffic continues to climb, our platform automatically launches more instances - two, four, eight or even more - while response times remain exactly the same, no matter how many requests we receive each second.

Traffic patterns when trending

Business as usual

The Divio platform is especially well suited to handling unexpected surges in web traffic, through being able to automatically manage underlying cloud vendors and provision and scale resources accordingly.

In a more traditional setup, web servers sit behind a load balancer that distributes traffic evenly across each server, depending on their load.

Divio pairs each app on the platform with an app controller. The app controller and load balancers work hand-in-hand to make more intelligent traffic management decisions. The app controller monitors the behaviour of applications and reports back health to the load balancers. If an application begins failing for any reason, a new instance of the application is automatically deployed and put into service.

This strategy means that response times remain constantly low, and during traffic surges, users experience a fast and responsive site that can scale according to traffic.

A thousand new websites in four hours

We face a unique challenge: sometimes, when we get to pole position on Hacker News, we see a huge spike in traffic that makes particular demands on the infrastructure, and not just more requests for static content. If the news item is about a major new release of django CMS or Wagtail CMS for example, a very large proportion of visitors will want to try a live demo of the new release.

A django CMS or Wagtail live demo isn’t just a few pages on our website. Each instance is a completely new installation, a complete system. They are containers running in Docker, and each one has to be deployed. A deployment creates new Docker containers, building each from standard and custom images, and finally hooking it into our deployment architecture so that the fresh django CMS or Wagtail instance it hosts can be made available on the web.

Every single visitor who hits the Try a live demo link gets their own individual demo site, running on its own containerised environment.

As far as resources are concerned, each demo is much the same as a customer project on our system: everything required to deploy, host and manage a website. The only real differences are that we don’t provide separate Live and Staging server instances for demo users, and that demos last for 15 minutes, not indefinitely.

What’s more, we deploy these demos on exactly the same infrastructure as our regular customer projects - despite the sudden multiplication of load, we can afford to do that without a risk that performance will be affected, because the infrastructure scales efficiently and seamlessly.

When a user launches a new project, whether a demo or an actual customer project, Divio:

• Acquires the images for the chosen app stack

• Builds the containers

• Runs the automated processes to install platforms and frameworks - such as Django, django CMS, and numerous other Python packages (from our own pip servers), and also various binaries

• Runs through the deployment process, creating the new project, running all of its migrations, collecting static files and so on

• Runs the frontend deployment process, using Gulp and npm to install packages, compile code, pre-process CSS and so on

Now let’s say that over a four hour period, one hundred thousand hackers see us mentioned at the top of Hacker News, and that one in ten of them actually hit the link to visit our website, and that one in ten of those go on to try out the demo.

That’s one thousand demos to be served - one thousand complete instances, the equivalent in effect of a thousand new websites and a thousand new servers to be built for a thousand new customers - over four hours.

It’s critical for us that our demos, like all our other operations, consume the minimum resources possible. We want computing power, bandwidth and memory resources to go towards making users’ interactions with the system faster and their sites more resilient, not towards unnecessarily-hungry deployment processes.

To give you an idea how aggressively we chase efficiency gains: it used to take about 14 minutes for a Divio deployment. Since then, without throwing more resources at the system, and just by implementing efficiency improvements, we’ve been able to lower that to around 1 minute. For demos, we’ve been able to take advantage of additional optimisation, and make each demo deployment faster still.

In our four-hour period at the height of Hacker News stardom, that’s a saving of 13 thousand minutes of processing time.

But still, from start to finish, an Divio deployment of a django CMS or Wagtail demo site takes under a minute. The patience of a web visitor is measured in fractions of a second, not tens of seconds. Quite obviously, a visitor from Hacker News is not going to wait a minute to try out a demo, and in fact, our demos are available instantly.

The secret sauce

So, what we do is have a pool of demos waiting ready to be allocated to visitors, and a system that will deploy new demo instances to the pool as they are being consumed.

Thanks to the efficiency improvements our engineering team have been able to procure in the system, we’re now in the happy position of being able to provide almost unlimited numbers of free demos without needing to worry about either being able to meet the demand, or the resources required to meet it.

The next time news of a django CMS or Wagtail release gets to the top of Hacker News and a thousand Django fans launch their demo the web analytics mountains can peak as savagely as they like, because behind the scenes the infrastructure can spawn new instances to match them without breaking into a sweat.