One of the most surprising lessons I’ve learned since joining SOASTA is that the data analytics world is still learning some difficult lessons – the same difficult lessons that distributed systems development learned a decade ago. One of the most interesting of these is the effects of Conway’s Law on the way data systems are designed, and the resulting roadblocks that can place on effectively solving problems unless it is addressed.
[O]rganizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations.
Here’s an example:
Let’s say your organization has a business unit that relies on IT for data analytics technology platforms, but in no way discusses the actual business analysis and its context with their technical colleagues. Let’s also say that your IT department has extensive monitoring of technical performance, availability and security to make sure those systems are healthy at any given time.
Odds are there is (mostly) strong communication between the business and IT when it comes to requirements for the analysis platforms themselves, e.g. what types of analysis need to be run, at what scale, and with what query and visualization interfaces. However, almost as certainly, I would bet that there is never an exploration of how the technical data collected correlates with the business data collected.
Interesting questions are missed:
- How does our technical architecture affect our conversion metrics?
- Can we predict the cost of our cloud resource consumption for a given campaign?
- How should we prioritize programming and operations investment to increase revenue opportunities?
Digital business changes the rules
For most enterprise applications, these questions weren’t critical. The separation of concerns between the business unit and IT could survive as a vendor-consumer relationship within the limited context of feature specification and operations. However, the advent of business models built on software – the rapid transformation to digital business – changes that forever.
In digital businesses, the business is the technology, and vice versa. If the technology is unavailable, the business is unavailable. If the technology is slow and awkward, the business is slow and awkward. In digital business, as we often say, speed is money.
Just as important as this is the agility required to successfully thrive in an online business world. You can’t “lock things down” to control stability. You have to be able to change early, often and reliably, all while pursuing changes that increase the success of the business’s desired outcomes.
But, perhaps most importantly, digital business data becomes critical input to digital technology development and operations, and vice versa. Everything centers around one thing: the software users’ outcomes.
Communication between those designing and operating the business model and those designing and operating the technology system has be be aligned. And it is, in businesses that started from scratch as digital businesses. Netflix, Amazon, and Facebook all have formal communication processes between software development, operations and the business units that are responsible for profit and loss.
Furthermore, many of the “systems of engagement” created by enterprises were conceived, built and operated not by IT, but by the marketing department, so communication was within a organizational unit. Data sharing and correlation is second nature to these organizations.
Bridging the Conway divide
But as we see today, IT departments are gaining the skills and tools required to successfully deliver digital business systems through devops and continuous delivery. Which means it is now necessary to address the expand data communications between the business and IT to include metrics that help drive a unified agenda and constantly improve outcomes.
To achieve this, we need a common language – a set of metrics that is equally useful for business and technology operators, and that can be correlated to specific outcomes for each (and both) of those audiences. To me, that common set of metrics is all about user outcomes, as I noted earlier.
So here are my most basic recommendations for getting your organization to bridge the Conway’s Law communication problems in your digital business operation:
1. Measure user outcomes
My earlier post goes into why, but really there are three questions here: how does speed affect your user outcomes (e.g. conversion), how do user interfaces affect user outcomes, and how does content (e.g. marketing copy, product listings, articles, etc) affect user outcomes?
2. Collaborate on all of the data
Don’t make the mistake of trying to cram all data into a single data analysis tool, but bring the specialized tools you use every day together in one place where cross-function teams can monitor, analyze and discuss the data in real time. SOASTA DOC is great for this.
3. Correlate technology metrics to business metrics to drive prioritization
As you get a sense of how different data sets are related, you’ll begin to see ways you can use that information to prioritize investment and work in both the business and the technology context. A great example of this is SOASTA’s Conversion Impact Score, which helps identify which page performance improvements will garner the biggest impact on conversion and revenue.
Putting the above practices into play will allow your digital business to not only improve performance and availability, but also use data-driven insight to improve collaboration across your organization. In the end, the big winner from this practice is your users, which should have a direct impact on your bottom line.
- The now and future of Digital Performance Management
- How a conversation about outcomes can bring tech and business closer
- Meet the Three Musketeers of Digital Performance Management: Real, synthetic and virtual users
About the Author
James Urquhart is Senior Vice President, Performance Analytics at SOASTA, Inc. Named one of the ten most influential people in cloud computing by both the MIT Technology Review and the Huffington Post, and a former contributing author to GigaOm and CNET, James brings a deep understanding of disruptive technologies and the business opportunities they afford. James is a seasoned technologist with more than 20 years of experience in distributed systems development and deployment, focusing on service-oriented architectures, cloud computing, and automation. Prior to joining SOASTA, he held leadership roles at Dell, Enstratius, Cisco, Cassatt, Sun and Forte Software. @jamesurquhart