One thing that I’ve seen best-in-class web applications make use of is the ‘graceful turn-away page’, also known as the graceful deferral, turn-away, or throttle page. I apologize in advance for this post being a little light on technical content, but the reason for that is a throttle can be implemented in a number of different ways depending on application architecture. Therefore I’d like to just focus on why this should be a mandatory piece of functionality for any online application. I will talk a little bit about how this is done in some applications and the rest is up to you, the smart people, to decide on how to implement it for your application.
First things first – what is it? Simply put, it’s application functionality that throttles new users and requests flooding into an application while preserving the experience for people already on the site. Without queuing and throttling mechanisms applications that get more traffic than expected will normally blow up and impact everyone. Throttling will give new users coming in a graceful page saying something to the effect of ‘we are experiencing higher traffic that normal, please come back later’. It stops people at the front door of the store while letting everyone else inside the store finish shopping. The key here is that sessions already established and in flight are not impacted at all.
One of the best examples I’ve seen of this is for event registrations held by Active Network. Active Network does registration for things like marathons, triathlons and Iron Man events. These events typically have a fixed registration count but get many, many magnitudes higher users coming to the site trying to get one of those spaces. To further complicate matters, registration opens at a very well-publicized, set date and time. So the traffic profile is something like this: they have potentially tens of thousands of people sitting on the web site page clicking refresh while waiting for the event to open. At some point they flip a switch and let everyone in. First come first, serve on the seats available. At some point though they may have such an unexpected spike in traffic that they need to queue people at the front door and let people in the registration process get through with a good customer experience. If a spike like this happens, a nice friendly turn away page is shown to all new users. Performance is great for everyone still in the registration process. Brilliant.
I have seen this on some best in class web apps, but not nearly enough. I feel that this functionality should be mandatory for every site out there. Again, not just a friendly ‘fail page’ that everyone sees when things blow up. This should happen way before that. Any web server can show a branded error page. They key here is the throttling capability. There is usually some effort to make this work properly but it’s well worth it.
In order for this functionality to work properly it is usually implemented at either the application tier, the load balancer tier, or some combination of the two. The application server usually knows how many sessions are present in the application. Application servers like WebSphere and JBoss know how many in-memory sessions are present and can report this out over JMX. This is a great metric from which to base concurrent load levels. Web or application server threads or connections are another common metric to watch for making a decision on when to turn away. The load balancer is a great place to do the actual throttling because most modern load balancers, like F5 Networks BigIP boxes, can serve static html pages. Plus they are aware of how many connections are open to the web and/or application servers.
Does your web app have a graceful turn away or throttling mechanism to handle unexpected traffic? If not, I would highly recommend such functionality. Create it and test it to see that it works as intended. It doesn’t even have to be automatic. It’s okay to go out on a load balancer and manually put the page up if that’s what it takes. As long as it holds back the flood of traffic from the tiers that need to be running optimally to serve the existing customers then it’s a win. It will create a better customer experience for everyone during turbulent traffic periods. If you can’t keep the site from going down entirely, at least you won’t have a zero dollar revenue window because the whole thing blew up.
About the Author