The Performance Beacon

The web performance, analytics, and optimization blog

Deploying on the Cloud: Lessons Learned with CloudTest


Performance testing challenges have changed rather dramatically in the last decade. Release timelines are accelerating as delivery cycles become compressed. Speed to test has never been more important. In addition, consumer-facing applications require testing at greater scale than ever before, often against a production infrastructure. What hasn’t changed are the tradeoffs that testers often find themselves making in three areas: Time, Cost and Scale.

Ironically, these constraints have the greatest impact on the build and deliver aspects of performance testing.  The true purpose of the test – to identify, isolate and resolve performance issues – is usually lost in the unnecessarily complex process of preparing to run the tests themselves. As the scale and complexity of web applications increase, the challenges, when using tools that were not designed for today’s web and cloud environments, increase proportionally.


SOASTA’s CloudTest is a modern web application for testing web and mobile applications built to achieve massive scalability by distributing resources across virtualized (cloud) platforms. More importantly, CloudTest is built to deliver real-time analytic data from the often-immense datasets generated by web-scale load tests.

With CloudTest, the number of load agents is no longer a driving concern, as the entire cloud platform cost and deployment is on-demand and automated. SOASTA’s Global Test Cloud provisions grids of servers on Amazon, Rackspace, IBM, Microsoft, GoGrid and other cloud providers using a simple wizard-based interface.

For many commercial and Open Source options, each load server needs to be individually procured, deployed and managed for every test. In contrast to CloudTest’s minutes, deploying servers to cloud platforms could take tens to hundreds of hours per test session. Once the cloud-based load servers are deployed, many solutions force the choice between leaving servers running for the entire project duration (and thus incurring additional cloud costs) or taking them down and spending precious man hours to rebuild for each test session.

CloudTest automatically and quickly deploys grids of cloud-based servers to points across the globe. Once the user walks through the wizard, CloudTest takes care of the rest, with the test servers up and ready to go in a matter of minutes.

Without the grid wizard, load servers must be deployed individually, configured to communicate with the main controller, and managed if issues such as a locked or non-responsive cloud server arise. CloudTest eliminates the setup time and allows customers to focus on what matters – test execution and results. The chart below shows relative time to set up. It’s important to note that the CloudTest server deployments happen in parallel so the total setup time is not much more than setting up a single load generator, no matter how many servers are used.

Time Requirement Estimates for Cloud-Based Load Agent Setup



Other Tools

Validate cloud server operation/error checking


:05 minutes

Deploy Image


:05 minutes

Configure Load Agent Settings


:05 minutes

Connect and Validate Agents/Monitors, etc.


:05 minutes

Total per-server cloud setup time

30 sec. – 5 min.

20:00 minutes


SOASTA CloudTest dramatically minimizes the trade offs of time, cost and scale associated with test build and environment setup, shifting your focus to test results. The emphasis is on providing the highest-quality test execution, with real-time data, so rather than spending unneeded time and money on the people, tools and environment to simply start testing, your teams can test more often and take action based on immediately available performance intelligence.

SOASTA Marketing

About the Author

SOASTA Marketing

Follow @CloudTest