With Canada Day in the books, and after seeing two days of fireworks here in the USA’s oldest city — Saint Augustine, Florida — to celebrate the USA’s Independence Day, it’s time to get back to work.
But with the turning of the calendar on the summer’s first set of holidays, I start to think about another key set of “holidays”: Black Friday and Cyber Monday. And I wonder, what type of fireworks await? The good kind, with lots of visitors, great web performance, fantastic user experience and LOTS of revenue, or the bad kind, with outages, poor performance, bad user experience, low conversion rates, and not-so-hot revenue.
We here at SOASTA have been in the business of performance since 2006, and we have a great set of “lessons learned” — so great that they all can’t fit into this one blog post. I’ll pick one of the best of the best to share with you here so you can get back to work…
Traffic patterns are constantly changing
As my financial adviser is always reminding me, “Past performance may not be indicative of future results.” (He does know enough not to ask me to touch my Lowe’s stock — that will always have a special place in my portfolio!)
Just because you met last year’s high volume demands does not mean that you will be prepared for this year’s peak period. Both technology and culture conspire to make traffic difficult to predict:
- Since your last peak period, your site has changed, most likely several times a month, with continuous integration available, marketing/promotions, inventory, and underlying infrastructure.
- In addition, your user community and customer base has most likely changed the way they use your site. Their performance expectations are most likely higher as well.
So what do you do? You analyze — with your user data
You take the guessing game off the table. You don’t rely on sampling and fake data from traffic that is not your user base.
Before you say, “All right, let’s just jump in the pool and get started,” let’s take a look at how your customers are using your site.
- What are the most popular conversion paths?
- What percentage of your revenue is coming from each path?
- Does your current performance test plan cover these paths?
You probably will find that your conversion paths are not aligned with your test cases. Just because a test case was used last year does not mean that it should be a part of this year’s coverage.
Do not assume. Analyze. Below is a screen shot of conversion path analysis from mPulse, SOASTA’s real user monitoring (RUM) solution, and its Data Science Workbench (DSWB) component that provides this capability.
I’ll take this over designing test cases by flipping a coin, throwing darts, or reading log files. (Please allow me to roll my eyes the next time I hear someone using an “old” technology tell me that they picked their test cases by pouring over log files.)
The good, the bad, and the ugly
Unfortunately, not all data analysis is looking at all the “happy paths”. To get more “happy paths”, you need to understand which areas are causing your users problems. Which browsers, devices, page groups, bandwidth options.
Let’s check it out.
Below is a cluster analysis of an ecommerce site. The red dots, or clusters, represent characteristics of users. These clusters allow you to dig into performance issues that are being experienced by groups of users.
As an example, is the user experience from users in Aruba slow as a group compared to those in other areas? Does the red consist of a cluster of users from Greece using Chrome 40 on Windows 7 hitting the product page?
Armed with this level of data, you can not only adjust test planning and future test runs, but provide feedback to the development and UX teams, triage root cause, and adjust production to alleviate these problems, where possible.
The poster boys of bad behavior
Now what if I wanted detailed data on every aspect of my worst-performing user experience combination? That is, I would like to know the poor performers for network performance (e.g. might be a particular cable provider or network), and from what location (e.g. Gainesville, Florida, or Raleigh, NC, for example), device or desktop performance, operating system performance, performance against a particular landing page or page group, from a particular browser and browser version.
The areas in red below represent the dimension on your website where the bad behavior is originating from. Armed with this data, you can now target testing and analysis against known areas of concern that your real users are experiencing.
Our DOC makes house calls
Now that you are armed with all this proactive user data — thus enabling you and your team to effectively measure, monitor, analyze, react, and remediate as you lead up to your peak season — it’s time to put it all together in one place so that you can watch your upcoming event real-time.
Meet the DOC — SOASTA’s Digital Operations Center.
The Digital Operations Center (DOC) is a next-generation command center for your organization’s user experience and digital business. It gives you the ability to access information with extensive visibility across multiple external data sources, brings broader intelligence to your day-to-day operations, and enables collaboration amongst stakeholders across the organization.
The DOC is a platform that integrates all the live data streams and analytics relevant to your business’s performance, from the configuration of your load balancer and content delivery network to your sales campaigns and social media feeds, all within a single, unified view.
The platform consists of multiple displays that can be a continuous wall of displays, which is highly scalable to support a multitude of different configurations that can be managed from a single DOC controller in one location. Or it can be distributed amongst many locations and controlled centrally from a single DOC controller. The point is you have full control and flexibility over how you can implement the DOC in your environment.
Imagine viewing your event — live — and being able to proactively react to a behavior that’s trending towards red, but before it actually hits red? And most importantly, before your users abandon, before they hit social media to tweet about the bad experience that they’ve had, complete with snapshots of errors from your site, and before they close the transaction with your competitor never to return.
Performance engineering has come a long way since the 1990s, much like today’s fireworks. Flashy. More robust. Less back room. Think pyrotechnics. The definition of pyrotechnics is exactly fitting for the DOC: “a brilliant performance or display, especially of a specified skill.” The skill? Performance engineering and analytics.
Understand your users. Analyze their behavior. Adjust the performance of your website to match their behavior. Watch your revenue increase. Lather. Rinse. Repeat.
About the Author
Dan is the Vice President of Digital Strategy for SOASTA. In this role, Dan is responsible taking the world's first Digital Performance Management (DPM) solution to market as a trusted advisor for SOASTA's strategic customers, and changing the way ecommerce organizations approach the marketplace.