Where Does QA Fit in DevOps?

Fitting QA into a modern DevOps group

In a traditional software engineering organization, the QA group is often seen as separate from the Development group. Developers and testers have different roles, different responsibilities, different job descriptions, and different management. They are two distinct entities.

However, for folks outside the engineering team – say in Operations – they generally consider Development and QA to be in the same group. From this perspective those teams are working together to do a single job, with a single responsibility: deliver a product that works.

So what happens with QA in a DevOps organization? When Development and Operations merge together, where does that leave QA? How does the testing team fit into a modern DevOps group? This article will take a look at exactly that question.

The Reason Behind DevOps: Automated Deployment

When you look at the trend towards DevOps, it’s pretty clear that companies are adopting this organizational model in order to facilitate a practice of automated software deployment. DevOps provides the structure that enables teams to push software out as a service on a weekly, or daily, or even hourly basis. The traditional concept of a “software release” melts away into a continuous cycle of service improvement.

DevOps is Agile taken through its logical evolution: removing all the obstacles to getting high-quality software in the hands of customers. Once you have a smooth process for agile development and continuous integration, automating the deployment process makes total sense because it’s fulfilling the objectives that business managers crave:

  1. Faster time to market
  2. Higher quality

Increased organizational effectiveness

But before we move on, let’s ponder that for a moment: the purpose of DevOps is to get high-quality product out to the market faster – even automatically. The notion of “Quality” is built into the fabric of DevOps. If you couldn’t reliably push high-quality software out the door, DevOps would fail as a function.

Kongress_Feature image

Clearly, there is a critical role for QA in a DevOps group. So how are people fitting it in?

The Product IS The Infrastructure

We asked Carl Schmidt, CTO of Unbounce, what he thinks about QA and DevOps. Unbounce runs a SaaS solution for online marketers, making it really easy to build, publish, and A/B test landing pages without IT resources. Unbounce has three development teams, each with a resident QA team member, and practices DevOps throughout the organization.

Carl states, “I’m of the mindset that any change at all (software or systems configuration) can flow through one unified pipeline that ends with QA verification. In a more traditional organization, QA is often seen as being gatekeepers between environments. However, in a DevOps-infused culture, QA can now verify the environments themselves, because now infrastructure is code.”

The infrastructure is code. It’s a game-changing claim to any traditional development organization. Historically there was a clear separation between what constituted the product and what constituted the operation. You built a product, deployed it into a test environment where it could go through some quality control, and then eventually deployed it onto a live production system where real users could get at it. If there was a problem in product, the operations team had to fix it.

But as the lines blur between product and operation – as the very name DevOps implies – it’s no great leap to recognize that the environment itself is a part of the product.

Carl continues, “It’s QA’s responsibility to actually push new code out to production, so the DevOps team has been providing them with tools that make blue-green deployments push-button easy. Our QA team can then initiate deploys, verify that the intended change functions as expected, cut over to the newly deployed code, and also roll back if there is any reported issue.“

DevOps QA Is About Preventing Defects, Not Finding Them

QA takes a critical role in this organizational structure because they have the visibility and the directive to push code out when it is working, and roll it back when it is not. This is a very different mindset from QA organizations of 10 years ago, whose primary responsibilities involved finding bugs. Today QA groups are charged with preventing defects from reaching the public site.

This has several implications:

  • QA owns continuous improvement and quality tracking across the entire development cycle. They are the ones who are primarily responsible for identifying problems not just in the product but also in the process, and recommending changes wherever they can.
  • Tests are code, as any test automation expert will tell you. It’s a necessity, of course. If your process is designed to publish a new release every day (or every hour) there is no room for manual testing. You must develop automation systems, through code, that can ensure quality standards are maintained.
  • Automation rules. Anything that can be automated, should be automated. When Carl describes Unbounce’s deployment process as “push-button easy,” this is what he’s talking about.

Testers are the quality advocates, influencing both development and operational processes. They don’t just find bugs. They look for any opportunity to improve repeatability and predictability.

Beyond Functional Testing: Automation for Load Testing, Stress Testing, and Performance Testing

Now we are at a very exciting time in the transformation of QA, because while many organizations have mature processes for automating functional testing, they are only just beginning to apply these practices to other areas of testing like security and stress testing.

In particular, load testing and stress testing are critical disciplines for DevOps organizations that are moving at high velocity. A bottleneck introduced to a critical transaction process on an eCommerce website can bring a business to its knees. You want to do everything you can to identify scaling problems before a product is pushed out to a production environment, and you also want to keep a close eye on performance after it’s been released.

If you are curious to understand how your process holds up, check out this infographic: How Automated Your Performance Testing Is.

Creating a DevOps Culture

Unbounce CTO Carl Schmidt has some wise advice about his DevOps group: “we dislike using that term in favour of saying that we have a DevOps ‘culture.’” DevOps isn’t an individual, it’s a core value of a development organization. DevOps is more about trust, people, and teamwork than it is about process. It’s about the creating of software as an ongoing service, not a static product.

Although it’s not in the name (DevQuops, anyone?), the only reason that DevOps works is because quality is built into the entire system. You can’t get much more important than that.

Tim Hinds – Product Marketing Manager

Tim Hinds is the Product Marketing Manager for NeoLoad at Neotys. He has a background in Agile software development, Scrum, Kanban, Continuous Integration, Continuous Delivery, and Continuous Testing practices. Previously, Tim was Product Marketing Manager at AccuRev, a company acquired by Micro Focus, where he worked with software configuration management, issue tracking, Agile project management, continuous integration, workflow automation, and distributed version control systems.

Tim Hinds
Tim Hinds is the Product Marketing Manager for NeoLoad at Neotys. He has a background in Agile software development, Scrum, Kanban, Continuous Integration, Continuous Delivery, and Continuous Testing practices. Previously, Tim was Product Marketing Manager at AccuRev, a company acquired by Micro Focus, where he worked with software configuration management, issue tracking, Agile project management, continuous integration, workflow automation, and distributed version control systems.
Tim Hinds on Linkedin

The Related Post

Continuous Testing and Continuous Monitoring What is the goal of Continuous Integration? Is it to enable Continuous Delivery of the code developers’ produce out to users? Yes, eventually. But first and foremost it is to enable ongoing test and verification of the code. It is to validate that the code produced and integrated with that ...
LogiGear Magazine – December 2013 – Cloud Testing
LogiGear Magazine June Issue 2018: TESTING in DEVOPS
How to ensure a successful test-driven environment In order to ensure a higher quality product is released in the end, many teams have turned to test-driven development. Under this scenario, quality assurance metrics professionals first create various QA tests, and then software engineers code based on these tests, typically while using a robust enterprise test management ...
In this article, I share some of my experiences and observations on training teams, mainly in corporate settings. The operative word here is “team”, not “individual”. When training teams or groups in an organization, many of the considerations and benefits are different than those for the individual. We’ll examine those differences and I will share successful solutions. ...
If you haven’t already caught a glimpse, we announced in January the Testing in DevOps and Continuous Testing videos series, available on YouTube!
Run your TestArchitect API and headless browser tests inside Docker containers as easy as flipping a switch Docker is a virtualization platform enabling you to create containers – mini virtual machines— which have their own predefined environment, including file system, libraries and settings. Best of all, these light-weight images eat up only a few megabytes ...
Do you want to speed up your automated tests by a factor of 10 and deploy your application continuously? In this article we share how the JIRA development team at Atlassian has accomplished this using Stages in Bamboo. Stages have allowed the JIRA Development team to take a week’s worth of testing and condense it ...
Aligning the Dev and Ops Teams DevOps as a philosophy has had as its centerpiece the principle that Dev and Ops teams need to align better. This is a people and organizational principle, not a process centric principle. To me this is more important when adopting DevOps than any other capability or tool. My last post ...
LogiGear University announces the launch of a new, free video series on Testing in DevOps and Continuous Testing which is available today.
Over the years we’ve provided an extensive number of articles, videos, and infographics that provide a wealth of knowledge about Continuous Delivery.
For this month’s book review, I read Continuous Testing for DevOps Professionals: A Practical Guide from Industry Experts, by various authors and edited by Eran Kinsbruner. The book is divided into 4 sections: Fundamentals of Continuous Testing, Continuous Testing for Web Apps, Continuous Testing for Mobile Apps, and The Future of Continuous Testing. The Fundamentals ...

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe