Tag Archive for ‘Cloud’ rss

The Future of Testing: Evolve or Elimination. It’s a Choice.

Posted by Tom Lounibos on Nov 17, 2011. 1 Comment

I’ve been seeing a lot of tweets and blogging around the future role of the Software Tester.  Some have suggested that, with all the recent advancements made in test automation, the test community may face employment challenges. I strongly disagree. That said, rapid technology innovation necessitates that users adapt and evolve, and the test community will not be immune to this requirement.

The fact is that today, in 2011, the majority of Testers are still relying on the same tools that they used in 1989. How is it that this community has been not been forced to change all that much over the past 20 years? This is highly unusual and counterintuitive in an industry that has long been affected by term limits for technology, one that is known for discarding old tools in favor of new tools to support emerging technologies.

How is it that so many Testers still use tools borne during the Reagan Era and when client server applications dominated the tech landscape? Have our applications neglected to evolve? Of course not!

One theory is that Testers are just wired differently, that they are more resistant to the forces of change. If this is the case, consumers beware: the impact will surely lead to an increase in web and mobile app failures. Why? If Testers hold onto previous testing practices then today’s dynamic applications will need more and more frequent performance testing because of the demands for speed, scale or cost savings.

So what needs to happen?

Like other professionals, Testers must evolve their skill sets. They must acknowledge and respond to the pressure to deliver actionable intelligence back to the business as a result of their testing. Further, they must begin to understand and accept responsibility for all aspects of delivering performance. Performance is the correlation of Application, Infrastructure, Network and Users, and a valuable, savvy Tester will deliver aggregated and correlated results to support their actionable intelligence.

It’s a new world of HTML, gesture-based UOs, Cloud, streaming video and voice, and Testers must embrace the opportunity or they will find themselves out of work. Carpe Diem Testing. Adapt and Evolve. Join me in helping the testing community become part of the cognoscenti, be on the forefront of this technology adoption lifecycle, earn industry respect by embracing testing’s place in the revenue cycle, and be the leaders in delivering the next generation of High Performance Applications.

 

Will the Cloud Change the Industry Giants or Will the Giants Change the Cloud?

Posted by Tom Lounibos on Jun 01, 2011. Leave a Comment

Over the past couple of years or so the technology giants such as IBM, HP, Cisco, Oracle and Microsoft have all announced cloud strategies — all with promises of multi-billion dollar investments backing these strategies up. Some have even predicted that cloud computing is the key to their future over the next ten years. This type of affirmation from these industry leaders is pretty heady stuff for a cloud industry in its own infancy…but will it become a reality? Will the industry leaders of today’s technology market be the leaders of tomorrow’s?

Most people following this market will say “absolutely,” even though the majority of cloud innovation continues to flow from early stage start-ups in this space today. It is also clear that the technology giants have both the cash reserves and the desire to take a leadership position in the cloud through acquisition. But is this enough?

The biggest obstacle to the giants achieving their ultimate goal of owning the cloud may be in their existing business models. The very same business model that has been the envy of the industry for the past thirty years may, in fact, be a roadblock to owning the cloud unless something changes.

Companies such as IBM, HP, and Oracle have been built largely on top of enormous distribution channels of direct sales and support personnel to attract new and maintain existing customers. While these channels have been the envy of the industry for the past thirty years, they also come at an enormous cost that must be compensated for in these companies’ pricing models. Conversely, the cloud’s on-demand pay-per-use business model (often defined as the “anti-lock-in” model) may be the antithesis of the enterprise site license that was popular in the 80’s and 90’s. Most importantly for publicly traded companies is that the cloud’s on-demand model offers limited visibility into projected or future revenue streams, which is a major issue in delivering guidance to share holders. Even worse, the average deal size for on-demand servers ranging from $.10 to $.80 per server hour rarely, if ever, goes over $100,000, let alone the $1M threshold that commonly justifies the existence of a direct sales organization. So the problem is, how do these companies sell cloud services? Their existing channels are focused on $1M deals, so getting their attention on a $50,000 cloud services deal may prove to be problematic. Developing a new organization around selling cloud services may cause some major internal channel conflict.

These challenges may explain why there have been more announcements and declarations than actual cloud successes so far from the industry giants. That said, I would not bet against them, as cash is still king…but I suspect independent cloud divisions will have to be formed to achieve any quick success. It also means that lean and agile cloud start-ups that are evolving from the beginning around an on-demand model model have more of a fighting chance than start-ups in previous generations. I, for one, am looking forward to seeing how the cloud market plays out between the “Davids” and the “Goliaths.”

Consumer Websites Must Stop Cheating on Their Tests

Posted by Tom Lounibos on Nov 01, 2010. Leave a Comment

Another week goes by and more leading consumer websites crashed and burned, causing losses in both revenue (millions) and, perhaps more importantly, in consumer confidence in ecommerce. This week fail whales were sited swimming in the oceans of one of the world’s largest retailers, a leading children’s toy manufacturer and the industry’s leading online payment processor. PayPal alone (which was down for several hours this past Friday) may have lost hundreds of millions of dollars for itself and the enormous network of retail outlets that rely on them for their financial transaction services.

So why are fail whales continuing to happen so frequently? Don’t these companies test their sites? The answer, for load and performance testing, is of course they do…most of the time. In fact, some companies are spending millions of dollars on people, hardware and tools to load and performance test their websites. So why all the fail whales? One answer may be that organizations that have chosen the wrong testing tools or test service are, in fact, cheating on their tests!

Consumer facing website have become the primary channel of revenue and product information for millions of companies around the globe. So why cheat on testing? The pressure to deliver (business agility) is enormous today, for all IT organizations, and may well be a key reason cheating has become a common practice. Many test subcontractors and test companies cheat on their tests simply because they run out of time. According to PCWeek, this is what happened to AT&T on the pre-registration site for the iPhone4 launch. Due to a last minute feature upgrade they had no time to adequately performance test the site, which, of course, crashed an hour after it was launched, due to a 10x spike in traffic, creating a PR and revenue nightmare! Other organizations cheat because they just can’t afford the resources (people, hardware and tools) to properly test their sites in the first place.

The most common way to defend this cheating is to use semantics to obscure the shortcuts taken. When asked by an enraged business owner “did you test the site before going live?”, the answer is always “of course we did”. The problem is that it’s the wrong question. The right question is “did you test the site by accurately simulating real users performing both normal and unusual tasks, at and above expected volumes?”. For instance, if the goal was to simulate 5,000 concurrent users, a tester may respond that they tested for 5,000 “page views”! This is when language matters, since 5,000 page loads rarely equals the activity of 5,000 real users. In fact, it likely represents only a fraction of the target volume. By simply substituting page views or transactions for accurately simulating the activity of users on the site they almost certainly won’t reach the expected goal set by the business owner.

Another method of cheating the system is to adjust the timings of test scenarios. This practice is widely used by the testing community, primarily due to the cost of hardware and software when using traditional testing tools. For example, if buying a plane ticket online typically takes about 10 minutes, a clever tester may reduce the timing of this process in the test to just 1 minute. This, on the face of it, allows many more “users” into the system, but it doesn’t accurately simulate real world conditions. Finally, many leading edge companies are beginning to realize that the only way to accurately test a site is by including production testing. Testing only in the lab, and then extrapolating the results for the production environment, leaves far too many variables unaccounted for in the complex deployment environment that is the web. Again, cheating the testing system.

Whatever the reason for the cheating (lack of time, people, or resources) we must change this game now to maintain a high level of consumer confidence and continue to expand the growth of online commerce.

What Happens When Vendors Repackage Old Technology and Call it a Cloud Service?

Posted by Tom Lounibos on Oct 23, 2009. Leave a Comment

In an effort to remain relevant, some software vendors take marketing advantage of the newest hot technology fad at the expense of their own customers. Cloud computing (the newest hot trend) has definitely been defined and positioned by some traditional software and hardware vendor’s marketing organizations to meet their specific agenda, which usually means extending the life of an existing product or product line.  It has become a virtual “cloud rush” as to how many times “cloud computing” can be mentioned in product and marketing collateral. . . including collateral for products that were first developed back when Bill Clinton was president!

A great example of this “cloud rush” is Hewlett Packard with its LoadRunner product.  LoadRunner was developed in the early 90’s to help corporate development teams test client/server applications.  It became, over time, the de-facto standard testing tool for most enterprise companies and was priced accordingly.  Entry-level pricing began at $100,000 and if you needed to simulate thousands of users the cost skyrocketed into the millions of dollars very quickly.

Today, HP is attempting to “perfume the pig” so to speak, by repackaging LoadRunner into a new cloud-based service called Elastic Test. To the uneducated observer it simply looks like a new cloud service for testing web applications.  The problem is that it’s the same LoadRunner product built almost 20 years ago to test client/server applications and it carries a lot of technology baggage along with it. Subsequently, HP chooses to pass along a lot of this baggage in the form of costs back to its customer base.  For example, an entry-level HP virtual test will take weeks to develop and will have a “starting” cost of $45,000.  Hardly living up to cloud computing’s value proposition for on-demand services that provide ease, speed and affordability.

Newer players are quickly swooping in to fill in the gap left by HP’s lack of R&D in the cloud space. New players such as SOASTA, whose CloudTest service was built exclusively to leverage cloud computing for testing. It offers a low-cost, highly scalable and easy-to-use on-demand test model for customers.  As a comparison, the $45,000 “entry-level” HP Virtual Test that takes weeks to perform and deliver results, can be done within 24 hours at a cost of $750 using SOASTA’s On-Demand CloudTest service. This lower cost model also delivers greater quality testing. Traditionally, you may have only performed 6-8 performance tests on a client/server application because of the cost and time required.  Now, with true cloud testing services like SOASTA CloudTest, you can perform hundreds of tests for significantly less cost and in the same amount of time it takes to perform 6-8 traditional tests. This impact is significant, delivering greater reliability in your web applications.

Customers choosing to stay with HP as their test vendor out of loyalty will continue to have to deal with a 20 year old technology trying to conform to a 2009 eco-system, as well as pay a premium of up to 45X for that loyalty.  This, in a world where websites need more and more performance testing as they become more complex and we reach higher and higher spikes in user traffic.

Cloud computing is a “game changer” for customers seeking greater levels of web reliability. It enables new, leading-edge, agile and cost-effective cloud testing services.  However, be careful of impostors claiming to offer cloud services . . . more often then not you will be able to recognize them by their price tags and by the lack of quality in the actionable intelligence they deliver.  SOASTA CloudTest is the cloud leader in delivering the highest quality performance intelligence available on the market today.

Email Us!
Subscribe to our Feed!
Join us on LinkedIn
Find us on Facebook
Follow our Tweets
See our pics