So earlier in the week I posted a blog that discussed how the fear of testing in production originated and that the origin of this fear really came from performance testing products of that time period, chiefly, LoadRunner.
Many readers (who are also still LoadRunner users, apparently) messaged me privately on social media with several questions and, in some cases, challenges.
So, I thought, what a better way to follow-up the first blog than to address some of these questions and challenges in a follow-up blog.
And don’t worry, I will not reveal anyone’s identity, or that some of them admit that this is the only technology from that time period (read: 1989) that they still cling to, and will defend, though apparently not publically.
So, lesson #1: Mercury is no longer a planet. Mercury was bought by HP in 2007. So, this product is no longer Mercury LoadRunner. Let’s at least get the name right.
Now, on to the questions/challenges:
Question: What programming language does SOASTA CloudTest use? LoadRunner uses C, which is a very stable language.
Answer: Well, yes, “C” is very stable. It has been around since the 1970’s. I used to like using FORTRAN, but you won’t see me using that in 2015. SOASTA CloudTest uses visual scripting. Point-and-click/record. No need to set-up an entire software lifecycle around scripting like one of my readers has done. To this I say: Let’s have a script-off. Fastest one to test wins! SOASTA CloudTest will be testing while you’re still going through your static code analysis peer review process (Seriously? You really have to do code reviews?).
Question: With LoadRunner, I can pull test data into a single dashboard, does CloudTest support being able to pull different types of data into a single dashboard?
Answer: Yes, and we’ll even go one better. CloudTest offers a single integrated view dashboard, but CloudTest does not limit you to just one dashboard. When executing a test, different dashboards can be displayed with various types of data based on the role that the person watching the test desires to see (e.g., maybe a performance engineer wants to drill down into specific page metrics, so he/she would want to view the waterfall dashboard that LoadRunner does not have, or a developer wants to see specific application metrics).
Question: How does CloudTest’s ability to generate load from the cloud differ from LoadRunner?
Answer: Cloudtest can generate load from over 17 different cloud providers located in over 57 countries around the world. LoadRunner is limited in this area with the ability via Stormrunner to generate load from a limited set of Amazon cloud locations that certainly to not number the breadth and depth of SOASTA’s cloud grid capacity.
Question: There are times when I am running a test and I see that I would like to increase transactions to simulate more traffic because the current performance is greater than we expected because of the change that we are testing appears to have helped fix a previous bottleneck. How easy is it to re-start the test?
Answer: Why re-start the test? With CloudTest you would have the ability to ramp up or ramp down load in real-time, during a test. This feature, Dynamic Ramp, shown in the photo below, allows you to increase or decrease load by location and overall with the use simple dials on our globe dashboard.
And before I get a question or comment like, “Well, LoadRunner has a globe dashboard”….. I’ll quote Crocodile Dundee, who might have said, “That’s not a globe. This is a globe!”
Question: Not really a question, but I see the value of running a test in production, but my company has a policy that we do not allow testing in production. How has SOASTA been able to convince other companies the value of testing in production, and how did SOASTA help them make that transition?
Answer: SOASTA has run over 10 million tests since CloudTest was announced and launched in 2008. The vast majority of these tests have been executed in production environments. The biggest reason why we were able to do this fearlessly is because CloudTest provides real-time analytics on all aspects of the environment under test so if the test is about to “tip it over,” CloudTest provides the ability to proactively view how the environment is behaving, and, if it is about to “tip over,” you can simply pause, or stop, the performance test. SOASTA is able to do this for our customers with the “Best Practices” for testing in production that we’ve learned with performing over 10 million tests. We’ll teach you to do the same thing.
I could go on and on, but this is a blog, not an e-book.
Welcome to 2015. Let SOASTA bring you there with us.
Want to know more about testing with SOASTA CloudTest? Let us show you how we do it.
Performance IS everything. SOASTA.
About the Author
Dan is the Vice President of Digital Strategy for SOASTA. In this role, Dan is responsible taking the world's first Digital Performance Management (DPM) solution to market as a trusted advisor for SOASTA's strategic customers, and changing the way ecommerce organizations approach the marketplace.