The Performance Beacon

The web performance, analytics, and optimization blog

You can’t get there from here (and other myths about testing in production)

I grew up in New England. And back before Google Maps and GPS were available, if you needed directions, you stopped and asked the locals at the full-service gas station. Normally, they’d get you on your way, but there were those times where they’d look you up and down and you’d hear, “… well, son, you can’t get there from here.”

OK, well, then how DO I get there?

You Can't Get There From Here Blog Pic 1

For years, testing has been considered a second class citizen when it comes to the pecking order of importance in any endeavor. Whether the endeavor is e-Commerce related, or any other application, testing has been synonymous with other commodities. The words, “I am a tester,” might typically get a response along the lines of sympathy card from Hallmark. Oh, sorry to hear that. Keep plugging away you’ll make that leap up the hierarchy at some point.

With today’s complexity found in all areas of software, not just e-Commerce, but cloud, and just about anywhere that has software that in some way acts as the front end to some activity initiated by a user, testing has become a complex role that encompasses more than just an assembly line mentality of checking the box and moving onto the next item rolling down the assembly line.

That’s right. Today’s testing is not your father’s 1989 Oldsmobile. There is an “e” in testing. But it’s not the one that you are thinking of-nope. Today’s “e” is found in multiple areas that that today’s testing encompasses. Today’s “e” is Performance Engineering. Today’s “e” is User Experience. Today’s “e” is Data Science.

But first let’s have a little history lesson. Let’s start in 1989. You know, 1989. The debut of The Simpsons. The Honda Accord is the top selling car. The first Lexus is marketed by Toyota. The first GPS satellite is launched, and 23 more would follow over the years. And, Vice President Dan Quayle sends out 30,000 Christmas cards with the word “beacon” spelled “beakon.” But more on beacons later (little did Quayle know how big beacons would be in 2015!).

In 1989 a small start-up named Mercury launched a product called LoadRunner. It was designed to be a performance testing tool for client/server applications. Over the next 10 years, it becomes the best selling performance testing tool in the market, with over a 75% market share. This coincides with the boom of the client/server architecture that rises up to threaten the stalwart mainframe market. Businesses shift to a dependence on client/server as the main core of their business functions. Testing these business functions becomes critical to the business. Testing them with LoadRunner becomes the norm. All is right with the world (heck, even I spent time at Mercury).

Then, in 1995, Al Gore invents the internet (at least the commercial aspect, anyway).

Unfortunately there is one major feature lacking in LoadRunner: Real-time analytics to analyze what is happening in an environment that is undergoing testing by LoadRunner. This leads to words spoken by every IT Operations/production team since LoadRunner was first introduced in 1989: “We do not test in production.” Of course you wouldn’t. You wouldn’t want to drive a car without dashboards and metrics like speed (faster, faster!!), temperature, fuel levels, etc., so why would you even entertain the idea of doing the same thing on your companies business backbone? You wouldn’t. Exactly.

So, fast forward to 2008. A company named SOASTA enters the market with a product named CloudTest. A product with an analytics engine that enables the real-time view of the testing being conducted against the target environment. An analytics engine and real-time data view capability that enables complete visibility into the target environment under test. Only one problem. LoadRunner has created a fear factor barrier for the testing community that resonates in most IT Operations teams and production environments everywhere:

“We don’t test in production.” Fool me once, shame on you… Fool me twice…. Ah, no. But wait. Not so fast, my friend.

Gradually since 2008, companies, over 500 of them, have dipped their toes in the water and found that with SOASTA it is a competitive advantage to test in production. Many of those same companies still will not allow LoadRunner into production—if they still use them at all. Why? It’s easy: the real-time analytics. The ability to “see” in production during a test.

Oh, so to compete HP LoadRunner now has spawned StormRunner. As Crocodile Dundee might have said, “That’s not a globe. This is a globe!”

You Can't Get There From Here Blog Pic 2

CloudTest’s real-time view allows you to see virtual user data being generated in a test against the production environment, while also allowing a view into real users, complete with where they are coming from, what they are doing, and what their user experience is. Real data. In real-time. In production.

So what are all those dots on the globe you ask? Those are beacons (not “beakons,” Vice President Quayle). Real-user beacons. Real people. Real customers using your web/mobile site/application—in production. It’s real user measurement. In real-time. Really.

Oh, and what are the sliding dials? In CloudTest, you have the capability to increase and decrease the virtual user load dynamically during a live test. From each location. Or all locations. Stormrunner? Ah, no. Sorry! Can’t do that.

Welcome to 2015. Let SOASTA bring you there with us. Leave 1989 and LoadRunner in your rearview mirror where they belong.

Want to know more about testing in production with SOASTA CloudTest? Let us show you how we do it, or read about how we do it.

Download our White Paper on Testing in Production.

LoadRunner in production? Seriously? Ah, no. Stormrunner? No, thank you. I’ve had enough snowstorms this winter, I certainly don’t need the havoc of a storm in my production environment caused by LoadRunner. Fool me once…..

As the local New Englander’s would say at the full service gas station when you stopped looking for directions, “You can’t get there from here”….. Even with today’s 24 GPS satellites, Google Maps and smart phones.

CloudTest on the other hand. Well….

Performance IS Everything. SOASTA.

Dan Boutin

About the Author

Dan Boutin

Dan is the Vice President of Digital Strategy for SOASTA. In this role, Dan is responsible taking the world's first Digital Performance Management (DPM) solution to market as a trusted advisor for SOASTA's strategic customers, and changing the way ecommerce organizations approach the marketplace.

Follow @DanBoutinGNV