The Performance Beacon

The web performance, analytics, and optimization blog

Blame the Farmers for the Silos. We’re Just Fixing the Problem With CI & DevOps

I have always wondered how IT ended up with organizational silos. I came from an aerospace background, and believe me, everything was integrated. If you were building airplanes, you wanted all the systems to work together from day 1 so that nothing bad could happen (a failure there is loss of life, not someone getting paged while they are sleeping). Aerospace was where Integrated Product Teams (or IPTs) were born. All your domain experts on one team. For the entire lifecycle. Made sense.

So, imagine my surprise when one day while sitting in heavy traffic on my way to a client appointment that I finally saw it—right there in front of me. The origin of the IT organization. Taken right from the farm.

So now that I have someone to blame, how to we address this age-old problem while keeping the lights on? How do we get to the ideal DevOps assembly line?

DevOps Blog Image - 1

Easy. We look again to aerospace (after all, aerospace had GPS in 1981, and we just couldn’t tell anyone. “Top Secret” you know). Look at how aerospace built things.

They used IPTs. Sounds simple, right? It is, if you understand that IPTs also mean processes. Not all of them. Just the right ones.

Now back to our silos. Some processes belong in a silo. As an example, you would agree that software design, and coding, belongs in the first silo on the left, or development silo, right? And, that monitoring belongs in the last silo (when looking left to right, of course). No argument there.

Where it gets murky is in the middle silo—the traditional testing silo. And, in the processes that seem to have been lumped into that middle silo. Processes that really are “conveyor belt” support processes. I call them “The Big 3,” because they are the foundation for product development speed and continuity, and coincidentally, they also form what is arguably the baseline for DevOps. They are: Change, Release and Configuration Management.

I would argue that these three run the length of all three silos and are the foundation of the silos. I would argue this because this has already been settled. I’ll point everyone to the US Commercial Software IT Standard, ISO 12207. First published in 1995, long before the catchy DevOps phrase. This was an IT standard written by aerospace company contributors (yep, I was one of them, but somehow you figured that out, right?).

And herein lies the problem in organizations that I see: each silo owns their own piece, but no one silo owns the hand-off from the other. Make sense? It should. That’s where the barriers exist today. The hand-offs. It’s what DevOps is trying to address with technology. The focus should be on the processes. The “Big 3” processes.

Let’s pick on one, as an example. Many organizations have, or had, something called the “Change Advisory Board (CAB)” or “Configuration Advisory Board (CAB).” This group, typically found in the IT Operations silo, was tasked with the ownership of determining whether or not a change, with its associated configuration, was promoted (or released) into production (see why I call it “The Big 3?”). In this case, IT Operations owns the last piece, the promotion, but the early handoffs were not owned by IT operations. Development handed to Test, which then handed off to IT operations, and the CAB.

See the problem? Let’s just fix it with IPTs, right? Well, yes. But today we also have ways to enable those processes. We’ve also invented “cool” terms for the problem. DevOps. Continuous Delivery. Continuous Deployment. Continuous Integration.

The solution is really just enabling better hand-offs between the silos that have been in existence since the farmers went down that path years ago.

I would argue that the middle silo, the testing silo, also contains pieces that should be spread across the other 2 silos. Testing should also be considered a supporting process. Why? Testing needs to start in the development silo—in fact, in most organizations, it already does—with unit testing. It should also extend into the operations silo. Why? You should be testing in production. You want to see how your application really works, and how your users are seeing it. You can only do that in production.

If your tool prevents you from being able to do that, then get one that lets you do that. It’s that simple. The tool should not drive the process. It should be the other way around. So, if you using tools from 1989 and your process is newer than that, then it’s time to fix it.

So, maybe the testing silo really breaks out something along these lines:

DevOps Blog Image - 2

So now that we have focused on the key areas of pain across the lifecycle, what should they look like?

Well, with SOASTA, the processes to solve these issues look like the lifecycle loop below.

To follow the process around the clock:

A requirement change (can be a feature request, a bug fix, etc.) leads to a code change, which leads to a “check-in” using a CI solution, in this case the open source product Jenkins (which, as an aside, is integrated with SOASTA). This then kicks off a test suite, usually, but not always, from the cloud, where the tests are “run” iteratively (hence the runners), while ensuring that platform/device coverage is automated, results are provided (“the truth”), issues resolved, a baseline established, and feedback from real users in the production environment, then we start the process all over again with a new stream (if Agile), or release cycle.

DevOps Blog Image - 3


So now the three farm silos running these process scenarios with SOASTA and Jenkins look like a team all working together.

DevOps Blog Image - 4


Sounds like a well-oiled machine to me.

It is. This is how SOASTA’s engineering organization builds our software. With several thousand tests per day and hundreds of release cycles and feature streams, this automation cycle is the best practice that keeps the conveyor belt moving along (and, yes, we use our SOASTA software to test. The tools do support the process).

The “Big 3” with performance engineering built into these key conveyor belt processes. Makes perfect sense.

Your take-away: focus and fix the process first. If the tools you are using can’t support this process: get new ones. I’ll say it again: the tools should support the process, not the other way around.

SOASTA’s best practices have helped hundreds of our clients implement this process cycle into their organizations with the goal to improve bottom line performance with speed to market, quality improvement and a laser focus on performance engineering across the lifecycle—all while keeping a laser focus on performance.

Interested in DevOps and the best practices that SOASTA uses to build world class software and how your organization can implement these same best practices in your organization to ensure performance?

Let SOASTA show you how it’s done.

After all, at SOASTA, performance is everything.

Dan Boutin

About the Author

Dan Boutin

Dan is the Vice President of Digital Strategy for SOASTA. In this role, Dan is responsible taking the world's first Digital Performance Management (DPM) solution to market as a trusted advisor for SOASTA's strategic customers, and changing the way ecommerce organizations approach the marketplace.

Follow @DanBoutinGNV