Where Does QA Fit in DevOps?

Fitting QA into a modern DevOps group

In a traditional software engineering organization, the QA group is often seen as separate from the Development group. Developers and testers have different roles, different responsibilities, different job descriptions, and different management. They are two distinct entities.

However, for folks outside the engineering team – say in Operations – they generally consider Development and QA to be in the same group. From this perspective those teams are working together to do a single job, with a single responsibility: deliver a product that works.

So what happens with QA in a DevOps organization? When Development and Operations merge together, where does that leave QA? How does the testing team fit into a modern DevOps group? This article will take a look at exactly that question.

The Reason Behind DevOps: Automated Deployment

When you look at the trend towards DevOps, it’s pretty clear that companies are adopting this organizational model in order to facilitate a practice of automated software deployment. DevOps provides the structure that enables teams to push software out as a service on a weekly, or daily, or even hourly basis. The traditional concept of a “software release” melts away into a continuous cycle of service improvement.

DevOps is Agile taken through its logical evolution: removing all the obstacles to getting high-quality software in the hands of customers. Once you have a smooth process for agile development and continuous integration, automating the deployment process makes total sense because it’s fulfilling the objectives that business managers crave:

  1. Faster time to market
  2. Higher quality

Increased organizational effectiveness

But before we move on, let’s ponder that for a moment: the purpose of DevOps is to get high-quality product out to the market faster – even automatically. The notion of “Quality” is built into the fabric of DevOps. If you couldn’t reliably push high-quality software out the door, DevOps would fail as a function.

Kongress_Feature image

Clearly, there is a critical role for QA in a DevOps group. So how are people fitting it in?

The Product IS The Infrastructure

We asked Carl Schmidt, CTO of Unbounce, what he thinks about QA and DevOps. Unbounce runs a SaaS solution for online marketers, making it really easy to build, publish, and A/B test landing pages without IT resources. Unbounce has three development teams, each with a resident QA team member, and practices DevOps throughout the organization.

Carl states, “I’m of the mindset that any change at all (software or systems configuration) can flow through one unified pipeline that ends with QA verification. In a more traditional organization, QA is often seen as being gatekeepers between environments. However, in a DevOps-infused culture, QA can now verify the environments themselves, because now infrastructure is code.”

The infrastructure is code. It’s a game-changing claim to any traditional development organization. Historically there was a clear separation between what constituted the product and what constituted the operation. You built a product, deployed it into a test environment where it could go through some quality control, and then eventually deployed it onto a live production system where real users could get at it. If there was a problem in product, the operations team had to fix it.

But as the lines blur between product and operation – as the very name DevOps implies – it’s no great leap to recognize that the environment itself is a part of the product.

Carl continues, “It’s QA’s responsibility to actually push new code out to production, so the DevOps team has been providing them with tools that make blue-green deployments push-button easy. Our QA team can then initiate deploys, verify that the intended change functions as expected, cut over to the newly deployed code, and also roll back if there is any reported issue.“

DevOps QA Is About Preventing Defects, Not Finding Them

QA takes a critical role in this organizational structure because they have the visibility and the directive to push code out when it is working, and roll it back when it is not. This is a very different mindset from QA organizations of 10 years ago, whose primary responsibilities involved finding bugs. Today QA groups are charged with preventing defects from reaching the public site.

This has several implications:

  • QA owns continuous improvement and quality tracking across the entire development cycle. They are the ones who are primarily responsible for identifying problems not just in the product but also in the process, and recommending changes wherever they can.
  • Tests are code, as any test automation expert will tell you. It’s a necessity, of course. If your process is designed to publish a new release every day (or every hour) there is no room for manual testing. You must develop automation systems, through code, that can ensure quality standards are maintained.
  • Automation rules. Anything that can be automated, should be automated. When Carl describes Unbounce’s deployment process as “push-button easy,” this is what he’s talking about.

Testers are the quality advocates, influencing both development and operational processes. They don’t just find bugs. They look for any opportunity to improve repeatability and predictability.

Beyond Functional Testing: Automation for Load Testing, Stress Testing, and Performance Testing

Now we are at a very exciting time in the transformation of QA, because while many organizations have mature processes for automating functional testing, they are only just beginning to apply these practices to other areas of testing like security and stress testing.

In particular, load testing and stress testing are critical disciplines for DevOps organizations that are moving at high velocity. A bottleneck introduced to a critical transaction process on an eCommerce website can bring a business to its knees. You want to do everything you can to identify scaling problems before a product is pushed out to a production environment, and you also want to keep a close eye on performance after it’s been released.

If you are curious to understand how your process holds up, check out this infographic: How Automated Your Performance Testing Is.

Creating a DevOps Culture

Unbounce CTO Carl Schmidt has some wise advice about his DevOps group: “we dislike using that term in favour of saying that we have a DevOps ‘culture.’” DevOps isn’t an individual, it’s a core value of a development organization. DevOps is more about trust, people, and teamwork than it is about process. It’s about the creating of software as an ongoing service, not a static product.

Although it’s not in the name (DevQuops, anyone?), the only reason that DevOps works is because quality is built into the entire system. You can’t get much more important than that.

Tim Hinds – Product Marketing Manager

Tim Hinds is the Product Marketing Manager for NeoLoad at Neotys. He has a background in Agile software development, Scrum, Kanban, Continuous Integration, Continuous Delivery, and Continuous Testing practices. Previously, Tim was Product Marketing Manager at AccuRev, a company acquired by Micro Focus, where he worked with software configuration management, issue tracking, Agile project management, continuous integration, workflow automation, and distributed version control systems.

Tim Hinds
Tim Hinds is the Product Marketing Manager for NeoLoad at Neotys. He has a background in Agile software development, Scrum, Kanban, Continuous Integration, Continuous Delivery, and Continuous Testing practices. Previously, Tim was Product Marketing Manager at AccuRev, a company acquired by Micro Focus, where he worked with software configuration management, issue tracking, Agile project management, continuous integration, workflow automation, and distributed version control systems.
Tim Hinds on Linkedin

The Related Post

Disclaimer: This article was originally published in the LogiGear Magazine, but has been updated for the June 2020 issue. Leading an organization into Continuous Delivery is a daunting task that usually takes software development teams months if not years. This guide covers the main points that organizations need to consider to increase their chance of ...
Introduction Everything changes. It’s the only constant. The landscape of software testing is undergoing a fast and dramatic change driven by societal, business and technology changes that have placed software everywhere. Computing is ubiquitous. There is hardly anything today that doesn’t contain a CPU or information transmission capability, with software to drive it. From smart toasters ...
This post is part of the Pride & Paradev series With continuous deployment, it is common to release new software into production multiple times a day. A regression test suite, no matter how well designed, may still take over 10 minutes to run, which can lead to bottlenecks in releasing changes to production. So, do ...
DevOps may be the next big buzzword, but Test teams really need to focus on its little sister, Continuous Delivery If you pay attention to trends in software development—from the perspective of what some sophisticated teams are doing, what articles and books are being written, to conference topics, you may have noticed the tools being ...
…On what you need to know before making the transition to EaaS 1. What are the main differences between cloud-based environments and cloud infrastructure? An environment is a collection of infrastructure elements working in conjunction to enable an application stack to work. For example, a simple 3-tier application, with a web front-end component, a business logic ...
Aligning the Dev and Ops Teams DevOps as a philosophy has had as its centerpiece the principle that Dev and Ops teams need to align better. This is a people and organizational principle, not a process centric principle. To me this is more important when adopting DevOps than any other capability or tool. My last post ...
Continuous Testing and Continuous Monitoring What is the goal of Continuous Integration? Is it to enable Continuous Delivery of the code developers’ produce out to users? Yes, eventually. But first and foremost it is to enable ongoing test and verification of the code. It is to validate that the code produced and integrated with that ...
DevOps has been described as Agile on Steroids; DevOps has also been described as Agile for Operations/IT. I like both of those descriptions. Many organizations want Development, Test, and Operations teams to move to DevOps now. DevOps is a big topic, but DevOps is not the focus of this article. We will not be talking ...
LogiGear Magazine June Testing in Continuous Delivery Issue 2017
LogiGear Magazine June Issue 2018: TESTING in DEVOPS
If you haven’t already caught a glimpse, we announced in January the Testing in DevOps and Continuous Testing videos series, available on YouTube!
How to ensure a successful test-driven environment In order to ensure a higher quality product is released in the end, many teams have turned to test-driven development. Under this scenario, quality assurance metrics professionals first create various QA tests, and then software engineers code based on these tests, typically while using a robust enterprise test management ...

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe