The Performance Beacon

The web performance, analytics, and optimization blog

Milliseconds Matter. Testing

An athlete doesn’t just show up at the Olympics, or any high-level competition for that matter, and successfully contend. There are literally years of preparation involved in converting raw talent into world-class performance. For an event the scale of the Olympics the host city also spends years preparing, and for the recent concluding 2012 Games, often deemed the first ‘social Olympics’, the pressure to perform was clearly felt by those responsible for the official website.

Website performance, like athletic performance, can be measured … and milliseconds matter. We are proud that earlier this year those responsible for the London Olympics website came to SOASTA. Up to that point they had been performance testing, but had no way to achieve the scale or distribution of load they knew they’d see during the Games. After the fact, we know exactly what they needed to handle:

  • 1 billion requests per day
  • 32 million visitors per day
  • 15 million of those on Games Time Website
  • 16 million on the mobile results application

How did they get there? While they‘d been testing up to 12,000 requests per second with an existing legacy solution, the team knew they needed to do more to be ready. Simply, they needed to generate load from a much wider geography and to a much larger scale. So they came to SOASTA. The CloudTest tests were built to make requests for various pages and assets to approximate the expected traffic. Initial testing went directly against the two origin data centers, separated into tests by data center, and then by desktop and mobile users. As the site was tuned, traffic was increased, eventually hitting 58,000 requests per second to each data center. This represented more requests per second than a complete cache refresh from the CDN and was helping to build the confidence of the LOCOG team in their implementation.



Along the way there were no dramatic finds, but at various load levels a number of different issues were surfaced – any one of which could have proven disastrous in production – prompting changes to load balancer and firewall settings, Apache settings on the forward facing web servers and fine tuning on the cache time-outs. One of the more perplexing issues was the disparity between the performance of the origin data centers, which in theory were meant to be exact replicas of each other. The graphic below show the disparity. After much investigation this was resolved by load-balancer updates.



Beyond the success of the origin tests, it was still important to test the infrastructure end to end. The effort culminated by testing 6 million unique users every 5 minutes during a 2-hour test. This test was executed twice. The virtual user would spend 60-90 seconds per visit cycling through a set of random pages on the site, and then start again. But it wasn’t just simulating scale that was important; the virtual users came from 15 locations and 450 servers around the world using SOASTA’s Global Test Cloud. The test needed to represent actual users, and so included Akamai and simulated traffic from desktop, iOS, Blackberry, Android and Windows Mobile devices.

After the global testing, there was a bit more work carried out on data center failover and resilience, under load. The preparation was done, confidence was high and the world enjoyed the 2012 Olympics without delay.

SOASTA Marketing

About the Author

SOASTA Marketing

Follow @CloudTest