Tag Archive for ‘functional testing’ rss

Traditional Vendors Out, New Players Emerge in Cloud Testing

Posted by Tom Lounibos on Jan 30, 2009. Leave a Comment

Every recession over the past 30 years has proven to be a forcing function for significant technological change.  In good times, corporations can get a bit careless or lose focus. However, in bad times, corporations go to great lengths to find new levels of cost efficiency in their operations . . . even if it means changing processes and technology that they had used for many years.  This recession is no different, change is beginning to happen, and traditional software vendors are the prime targets. The traditional “License” model employed by traditional software vendors is officially dead with this recession. The “Software as a Service” business model or even more specifically, the “Pay for Use” model, have emerged as clear winners.  IDC recently raised its 2009 projections for the SaaS market. They now expect this segment to grow by more than 40% this year.  This business model shift, coupled with the emergence of the new deployment platform of Cloud Computing will make it very difficult for traditional vendors to maintain their thrones.

Take my business, the test marketplace.  Our market was long dominated by Mercury Interactive which was bought by HP several years ago.   Mercury’s LoadRunner was the ideal tool for testing client-server applications in the late 80′s and 90′s, and its success drove Mercury to its leadership position under a license-based model.  However, a license model that was very expensive ($30k/Test Hour). Today, the world has changed for HP/Mercury. We are all now developing and deploying new and much more dynamic Web applications for consumers around the world. Mercury’s technology, business, and deployment model are looking a bit like Tom Jones performing at a Beyonce concert . . . a little out of place. New test vendors are beginning to emerge as the new leaders of the Cloud Testing market.  This includes new players like SOASTA that deliver a “pay only for test time used” ($1k/Test Hour).  Companies like these are greatly reducing the cost of testing while even enabling more and better quality testing. This is an example of another changing of the guard, made possible by another down economic period and a requirement for greater reliability.

contact me at: tlounibos@soasta.com; twitter.com/lounibos

Genentech Tests Cloud Computing

Posted by Tom Lounibos on Jan 29, 2009. Leave a Comment

As Michael Liedtke of Associated Press reported last week, and subsequently now we can also report, that indeed Genentech has migrated its 16,000 employee’s to Google Apps for its email and for some office suite applications. However, the more interesting back story to this article was before they made the decision, Genentech wanted to ensure that Google would scale. Google . . . scale?  That’s right they questioned what kind of user experience there employees would have when accessing applications deployed in a cloud. To get comfortable, Genentech decided to do some Cloud Testing using SOASTA’s new On Demand CloudTest Service. Over the course of a couple weeks Genentech were able to test several different user scenarios including having 16,000 simulated Genentech employees hitting Google Calendar at the same instant.  The testing went well, and the rest is history. Genentech has dipped their big toe into the Cloud Landscape, and by all the feedback we have received,  it went very well.

contact me at: tlounibos@soasta.com; twitter.com/lounibos

Cloud Computing is Changing How we Test Web Sites Forever!

Posted by Tom Lounibos on Jan 28, 2009. Leave a Comment

Okay, I know I’m biased, but as the rest of the blogging world focuses their attention on the “Cloud Platform Wars”, debating endlessly on which Cloud platform vendor will emerge as the “winner” in the coming years, quietly, several companies (from young start-ups to enterprise class) are leveraging the access, availability and affordability of Cloud Computing for testing their Web sites.  Even more amazingly, they are testing without even considering which Cloud platform they are using.  They are Cloud Testing!

Load, Performance, and Stress testing Web sites has been a black art of Web development shops for years. . . requiring a huge amount of compute power, expertise, and expensive system and test software. The total cost of Performance testing has reached an estimated $30,000/test hour. . . making testing costs prohibitive for most companies. That is, until Cloud Testing emerged in 2008.

Now companies are leveraging the limitless power of Cloud Computing to simulate Web (scale) traffic to test their sites before going into LIVE production. They are greatly reducing the occurrences of performance related errors, latency, and actual site crashes while reducing the cost of testing down to $1,000/test hour.

So, while the rest of the world tries to figure out whether or not Cloud Computing is for real, we are seeing many customers seeking a higher level of reliability for their web sites,  at a lower cost, finding a real value proposition in Cloud Computing.

Cloud Testing is Changing How we Test Web Sites Forever!

contact me at: tlounibos@soasta.com; twitter.com/lounibos

If Cloud Computing is The New Destination, Then Where does the Journey Begin?

Posted by Tom Lounibos on Jan 23, 2009. Leave a Comment

Which Applications You Should Move to the Cloud

Over the past year, there has been much discussion over the cost benefits of Cloud Computing.  Companies such as Amazon, Rackspace, Google, and SalesForce all introduced Cloud platforms in 2008 that stirred our imaginations and our hopes of a more cost effective and efficient way to deliver Web applications. By virtue of the amount of vendor activity and general buzz, it is pretty clear that Cloud Computing may be the right delivery platform given our current economic environment.  That said, it also has its critics, who site that using the Cloud as a platform for “real world” enterprise applications has some issues.  Many believe that it is unproven in the areas of data security and platform stability and reliability.  But even to critics it remains a compelling alternative.   So, if Cloud Computing is at the very least “a” destination platform for delivering Web applications…what types of applications make the most sense to make the journey to the Cloud?

Cloud Testing vendor SOASTA, has been in a very unique, first-hand position of observing how early adopters of the Cloud have made their decision as to which applications they would move and why.   For the past six months, SOASTA has been Cloud Testing hundreds of Web applications from a wide variety of industries such as consumer products, mobile, social networking, and financial services.   These companies range from small start-ups to the enterprise (i.e. Hallmark, Genentech, Proctor & Gamble, Qtrax, Pelago)—all seeking ways to reduce costs of delivery by using the Cloud.

From this experience, we see the following types of applications as the forerunners in moving to the Cloud:

Sales & Marketing Applications: these are applications that center around a one-time activity such as a marketing campaign for a new product or the preparation for a holiday (i.e. an influx of e-cards or flower delivery on Valentine’s Day).  A recent event that could have benefitted from Cloud Testing was the US Presidential Inauguration. Hundreds of millions of people turned to the Internet to watch the event, causing many site outages or failures.  The characteristics of this type of Web application that make them Cloud-worthy are:
a.    Event-driven, tied to a specific time period or date
b.    Large Load (Scale), Web traffic is unknown and must be met
c.    Variable Load, prone to web traffic spikes or surges
d.    Global, used by a widely distributed audience
e.    Media Rich, employ a high level of dynamic content such as Ajax or Flash and subsequently may require more compute power

Composite Applications: these are applications that typically aggregate data services such as RSS feeds from many sources into one application. Common composite applications include Priceline, NASDAQ and Facebook. The Cloud has proven to be a very a low cost, easy-to-use, aggregation and deployment platform.

Collaborative Applications: these are applications that have many of the same attributes the previous two applications.  These are typically shared or group applications with high potential for scale and spikes in Web traffic, global users, and dynamic content. They often require shared access and availability to large amounts of compute power. Because they are not typically revenue-generating applications, they are ideally suited for a low cost delivery platform such as the Cloud.  Salesforce.com, eBay, Youtube, and some Wikis, fit this category.

The final analysis suggests that every application has unique deployment requirements and that Cloud Computing offers many companies a low cost alternative.  SOASTA’s own application, CloudTest, leverages Cloud Computing to simulate real world Web traffic for testing Web applications and networks. Testing has less of a requirement for secure or portable data than most transaction-oriented applications. It negates some of the common concerns and is ideally suited for the Cloud.  In the end, companies should choose the deployment platform that fits their application’s specific requirements and not judge every application as having the same requirements.

Cloud Computing is quickly establishing itself as a viable delivery platform. The real question remains…for which applications?  Proving once again that it’s not the destination that matters, it’s the journey.   Let the journey begin.

contact me at: tlounibos@soasta.com; twitter.com/lounibos

Web Performance is Customer Service

Posted by Tom Lounibos on Jan 23, 2009. Leave a Comment

Cloud Testing is a Solution to the Problem


Fact, there are over 162 million registered Web sites today, yet only 15% (estimated) of them have had any actual Load Testing done to them. Which means all of us consumers are being used as the digital version of “Crash Test Dummies”. You have to ask yourself . . . doesn’t service matter to Web 2.0 companies?

Surely, companies like Amazon, FaceBook, Apple, and Netflix care about their Internet customers as much or more than traditional storefront companies do. After all, most of the time these are their “only” customers.

So why would they risk a customer experience that leaves a potential user/customer without service for hours, or perhaps worse waiting for 20 seconds as a new page is downloaded? Every year Business Week publishes its Annual List of “Top 10 Companies for Customer Service”.  Lists like these typically highlight traditional retail companies, and focus their evaluations on the performance of their sales or customer support telephone staffs. But, as we all move to the Internet to buy our goods and services, what happens to customer service on the Internet?

Given that the Internet is largely a self-service business, you may question just what  “good service” is to a Web user. Latency may be one of the barometers as to what defines what we should expect as good service from our Web sites. A recent report states that Amazon estimates it loses 1% in sales if their site experiences more then 100ms in latency. In another report Google states it loses up to 20% of its traffic to a page if that page takes more then a .5 seconds to load. Then there are actual web site crashes . . . companies like Skype have had very public crashes which there site crippled for hours.

So why are these sites failing, haven’t they been tested? Well, yes and no. They have been tested functionally, but very few companies actually do stress testing of their Web sites and applications. The problem is that it has become virtually impossible and too expensive to simulate or exceed real-world Web traffic in an internal test lab. Thus, Load Testing of Web sites, if done at all, is done in a very limited way (i.e. on average companies test less then 10% of there expected web traffic). Which means for all of us Internet surfers, strap on your Crash Helmets, and prepare to be frustrated with your carrier, your device, and with your service provider as you’re about to become the main resource testing how well their web sites perform.

But don’t despair too long. A new way to simulate real world scenarios is here and it’s called Cloud Testing. This new way of testing Web applications and networks leverages Cloud Computing to simulate Web traffic, and does so at a fraction of the cost of traditional testing methods. What used to cost millions of dollars now costs a few thousand for a few hours of testing. It’s fast, easy, and affordable and provides great insight to how applications will actually run in the real world. n the future there should be a new list of “Top Services Companies”. These companies should be recognized for as having the best customer service record for the Internet. That will be largely tied to their web performance. I suspect the winners will be using Cloud Testing.

contact me at: tlounibos@soasta.com; twitter.com/lounibos

Email Us!
Subscribe to our Feed!
Join us on LinkedIn
Find us on Facebook
Follow our Tweets
See our pics