Archive for March 2011

Cover Your Bases When Performance Testing

Posted by Dave Murphy on Mar 31, 2011. Leave a Comment

With the release of CloudTest Pro, SOASTA has reinforced a central theme of our Performance Test Methodology: expand your scope to include tried and true traditional methods of testing with the benefits of cloud testing: scale, speed, accuracy and flexibility.  CloudTest Pro makes it much easier to execute both internal and external testing without sacrificing control over your testing processes.  Also, it’s not just a matter of where you generate the load, you must test with a purpose.  CloudTest Pro CloudTest Pro makes it much easier to execute both internal and external testing.  We believe it’s not enough to simply measure end user experience or find out when bandwidth is saturated.  Important information?  Of course, but hardly the only keys to confidence.

To gain confidence you need to identify the things that are in your control and that you can address.  Can you improve the performance of the ecommerce shopper who’s at the end of a DSL line on a 3-year-old computer while the Roku box is streaming Netflix?  Of course, but not by buying all of your end users faster Internet access and better computers.  It’ll be by doing the same things you’d do to improve performance for everyone: ensuring that your web pages, applications, infrastructure, network and third party providers are performing optimally.

The main tenants of the SOASTA’s Performance Test Methodology include:

  • Testing both in the lab and live, web-based applications in production
  • Leveraging the Cloud to test at both typical and peak traffic levels, from hundreds of users to millions
  • Responding to the acceleration of development cycle times by making agile performance testing a realistic alternative
  • Generating geographically dispersed load for a valid representation of real-world traffic
  • Generating both internal and external load and using both the lab and production environments for the most efficient and effective results
  • Analyzing Performance Intelligence in real time to speed problem resolution

We’ve advocated external testing because it’s important to understand the performance from different geographic regions.  It’s also the most realistic and valid way to generate user traffic.  A complete suite of testing includes tests designed to identify specific issues or establish baseline metrics.  Comprehensive testing will often include stress, spike, endurance, failover, capacity and break-point tests in addition to understanding response times.  Make sure you’re getting a 360 degree view of performance.  For more thoughts on the topic, check out this post from last year: What Does Performance Testing Mean? http://www.soasta.com/2010/05/06/what-does-performance-testing-mean/

Best Practice Architecture – The Graceful Turn-Away Page

Posted by Dan Bartow on Mar 17, 2011. Leave a Comment

One thing that I’ve seen best-in-class web applications make use of is the ‘graceful turn-away page’, also known as the graceful deferral, turn-away, or throttle page.  I apologize in advance for this post being a little light on technical content, but the reason for that is a throttle can be implemented in a number of different ways depending on application architecture.  Therefore I’d like to just focus on why this should be a mandatory piece of functionality for any online application.  I will talk a little bit about how this is done in some applications and the rest is up to you, the smart people, to decide on how to implement it for your application.

First things first – what is it?  Simply put, it’s application functionality that throttles new users and requests flooding into an application while preserving the experience for people already on the site.  Without queuing and throttling mechanisms applications that get more traffic than expected will normally blow up and impact everyone.  Throttling will give new users coming in a graceful page saying something to the effect of ‘we are experiencing higher traffic that normal, please come back later’.  It stops people at the front door of the store while letting everyone else inside the store finish shopping.  The key here is that sessions already established and in flight are not impacted at all.

One of the best examples I’ve seen of this is for event registrations held by Active Network.  Active Network does registration for things like marathons, triathlons and Iron Man events.  These events typically have a fixed registration count but get many, many magnitudes higher users coming to the site trying to get one of those spaces.  To further complicate matters, registration opens at a very well-publicized, set date and time.  So the traffic profile is something like this: they have potentially tens of thousands of people sitting on the web site page clicking refresh while waiting for the event to open.  At some point they flip a switch and let everyone in.  First come first, serve on the seats available.  At some point though they may have such an unexpected spike in traffic that they need to queue people at the front door and let people in the registration process get through with a good customer experience.  If a spike like this happens, a nice friendly turn away page is shown to all new users.  Performance is great for everyone still in the registration process.  Brilliant.

I have seen this on some best in class web apps, but not nearly enough.  I feel that this functionality should be mandatory for every site out there.  Again, not just a friendly ‘fail page’ that everyone sees when things blow up.  This should happen way before that.  Any web server can show a branded error page.  They key here is the throttling capability.  There is usually some effort to make this work properly but it’s well worth it.

In order for this functionality to work properly it is usually implemented at either the application tier, the load balancer tier, or some combination of the two.  The application server usually knows how many sessions are present in the application.  Application servers like WebSphere and JBoss know how many in-memory sessions are present and can report this out over JMX.  This is a great metric from which to base concurrent load levels.  Web or application server threads or connections are another common metric to watch for making a decision on when to turn away.  The load balancer is a great place to do the actual throttling because most modern load balancers, like F5 Networks BigIP boxes, can serve static html pages.  Plus they are aware of how many connections are open to the web and/or application servers.

Does your web app have a graceful turn away or throttling mechanism to handle unexpected traffic?  If not, I would highly recommend such functionality.  Create it and test it to see that it works as intended.  It doesn’t even have to be automatic. It’s okay to go out on a load balancer and manually put the page up if that’s what it takes.  As long as it holds back the flood of traffic from the tiers that  need to be running optimally to serve the existing customers then it’s a win.  It will create a better customer experience for everyone during turbulent traffic periods.  If you can’t keep the site from going down entirely, at least you won’t have a zero dollar revenue window because the whole thing blew up.

The Significance of SOASTA CloudTest Grid

Posted by Dave Murphy on Mar 02, 2011. Leave a Comment

With the release of CloudTest Pro, SOASTA’s customers now have direct access to one of my favorite capabilities as a SOASTA performance engineer: the CloudTest Grid. The CloudTest Grid continues our success in reducing the cost, complexity and time to results for performance testing websites and applications.

Like any other engineer, a performance engineer needs timely access to professional tools before they can add real tangible value; i.e. delivering evidence as to whether or not a website or application will meet target responsiveness and scalability requirements, and help identify bottlenecks and their root cause. Not only does the engineer need access to professional tools, the tools need to be suitable for the job.

Most performance testing tools require significant capital investment in licensed software and hardware infrastructure, as well as ongoing maintenance costs just to test what is typically a scaled-down version of a production environment. This may have been appropriate for the internally facing applications of the past where user numbers have been counted in the tens or hundreds. However today’s web applications are delivering the next generation of dynamic content to an extraordinary number of global users every day. Keeping pace with the speed and agility of today’s web development initiatives requires a new generation of testing and performance management solutions.

Before joining SOASTA I found that it was becoming increasingly difficult to help organizations justify investing in traditional performance testing tools simply due to the cost and complexity of these offerings. Often, by the time a business case had been constructed and a flexible commercial model agreed upon the initial requirement for testing had passed.

These challenges have led to cloud testing as a way to reduce costs and provide the benefits of scale and more accurate testing. However, there remained the challenge of how to best leverage the distributed nature of the cloud and optimize deployment to be as efficient as possible.

With CloudTest Grid, performance engineers can deploy hundreds, or even thousands of load servers around the globe in a matter of minutes. The servers are automatically spread across locations and cloud providers to meet the requirements of each specific test, and the Grid takes care of all error checking and fail-over to replace any bad instances or get additional capacity from other locations, as needed. Once deployed, the servers are monitored throughout testing to track their health and ensure the Grid is operating as expected.

With SOASTA CloudTest Grid it is possible to provision a complete test environment, when required, and just for the period of time that is needed, which significantly reduces the time to test. We routinely perform tests involving tens of thousands or hundreds of thousands of simulated real users. Combined with our industry leading performance analytics SOASTA CloudTest delivers increased business value over traditional testing tools at a fraction of the cost with significantly improved time to value.

Email Us!
Subscribe to our Feed!
Find us on Facebook
Follow our Tweets
See our pics