The Performance Beacon

The web performance, analytics, and optimization blog

When Do You Know That it’s Time To Change . . . Vendors?

All good things come to an end at some point, but knowing how to recognize when it’s time to change can be difficult.  In sports, great athletes like Willie Mays and Joe Montana hung on a little too long and ultimately damaged their legacies in doing so. In technology, this can also be true as companies sometimes tend to hold on to their legacy technology a little too long thereby hindering their ability to compete. There are many reasons for this but at the core, any change can be difficult. But as in sports, the game does change, and you if you don’t react in time you potentially find yourself on the outside looking in.

When it comes to load testing, for the past two decades the market (largely enterprise) has been dominated by a single testing vendor, Mercury LoadRunner (now HP LoadRunner).  It was a terrific product when it was introduced over twenty years ago as a testing tool for internal client/server applications. It was widely adopted by the enterprise technology community as the standard for testing client/server applications. Unfortunately for HP, who acquired Mercury Interactive in 2003, the game has changed. Over the past five years, the market has shifted away from physical storefronts and moved toward building new, highly scalable and dynamic web sites as a lower cost distribution channel to reach the emerging global markets. This has left Loadrunner’s client/server architecture and expensive pricing model looking somewhat outdated as a testing solution for today’s new applications (which are primarily services). These services are being built with a whole new set of scalable technologies and are being deployed more frequently on a new platform called cloud computing.

With the markets changing, and an economy in turmoil, it may be time to reassess your existing technology vendors. Lets take a look at the testing space, specifically if you are an enterprise company using LoadRunner as your testing tool for the past several years. Is it time to change direction? The following five tenets can be used as guidelines to determine if it’s time for a change.

TEST QUALITY: If you find yourself testing only a fraction of your expected load due to the price and complexity of your current testing solution then this could be red flag. For example, if you are only testing 2,ooo virtual users (vusers) and you have the potential for up to 100,000, you are not really testing your site at all.   PRICING: If the reason you aren’t testing the full load potential is because the cost of testing is just too expensive (costs include hardware, software and people) then you may want to consider a shift to Software as a Service (SaaS) which offers a high quality low cost alternative.

TEST SCALE: LoadRunner was not built to leverage an elastic and virtual platform like cloud computing. HP claims that it will one day run in the Cloud, but if it does, it will not run well in the Cloud.

INNOVATION: Even if HP promises innovation to meet the explosion of new technologies that we are all using to build web applications these days, it will not come quickly. The original Mercury development team (who know where all the bodies are buried in the LoadRunner code) are long gone from HP. Finally, ARCHITECTURE: Why would you ever want to use a twenty year old testing tool, built to test client/server applications, to test a new web service when you could be using a leading edge, cloud-based testing service to test your new web services?

Bottom line, after two decades of using the same testing tools, it’s time to change to a new generation of cloud testing services.

contact me at:;

Tom Lounibos

About the Author

Tom Lounibos

As CEO of SOASTA, Tom brings more than 30 years of experience building early stage software companies, leading two companies to successful IPOs. Tom is a regular speaker at both cloud and testing events, and has become a leading advocate in using the cloud to empower individuals and accelerate changes in how applications are built, tested and deployed. Most recently, Tom served as President and CEO of Kenamea. Prior to Kenamea, he was CEO of Dorado Corp., a financial services software provider. Previous to Dorado, he was EVP of Sagent Technology through its 1999 IPO, entrepreneur-in-residence at Crosspoint Venture Partners, and held executive positions at Digitalk Corp., Knowledgeware (KWI) and Encore Financial Services. Tom also serves on several boards in the Silicon Valley.