Reasons to Consider Software Tests as Products


With DevOps, automated tests have become a crucial necessity. Tests need to be thorough, and their automation should be stable. In fact, tests have to meet quality and robustness criteria that are similar to the application under test, but tests seldom get the attention and investments that the applications get. Where sources and components of applications are considered products that are designed and developed, tests play a mere supporting role. In Scrum projects you will not see tests specified in the backlog. Rather, they are seen as a part of the production for the user stories.

There are some valid reasons to consider tests as products. Well-defined tests can help quality and velocity. In particular, test design is a major driver for automation success, something I frequently talk about in my tutorials at the STAR testing conferences. Tests have qualities like correctness, completeness, and efficiency that can be planned for and reviewed when done.

In our projects, we follow a modularized approach called Action Based Testing (ABT). Tests are organized into test modules that we manage as the products of test development. The test modules, in turn, contain objectives and test cases to match them. The test cases are written as a sequence of actions, spreadsheet format lines each with an action keyword and zero or more arguments. You can learn more about ABT here:

I also believe that there is such a thing as DevOps for automated tests.  Tests, in the ABT approach at least, have a development phase, which can be much earlier than the implementation of the actions they use. And when tests are correct and their automation is working well, they can be used for production, which is their execution as part of the development and deployment of the application under test (or its components). In particular, I believe it can make sense to have separate test modules to verify the more complex actions and interface mappings, before they’re executed in large-scale functional tests. To learn how to leverage ABT in DevOps, you can read this post written by Van Pham.

Another way of looking at tests as products is from a point of view of company value. If tests are well-designed with a clear scope and the right level of actions matching that scope, they can be reused across many generations of the application they’re testing. This is particularly true for business-level tests. For example, tests for mortgage calculations are dependent on business rules, not on the details of a UI or non-UI they’re running against. In fact, multiple banks could use the same set of mortgage tests. Having tests like that have value in themselves. They can, in theory, be on the assets side of a balance sheet. Allowing teams to develop them is an investment for the future.

This article originally appeared in Techwell:


Hans Buwalda is an internationally recognized expert in test development and testing technology management and a pioneer of keyword-driven test automation. He was the first to present this approach, which is now widely used throughout the testing industry. Originally from The Netherlands, Hans now lives and works in California as CTO of LogiGear Corporation, directing the development of what has become the successful Action Based Testing™ methodology for test automation and its supporting TestArchitect™ toolset. Prior to joining LogiGear, Hans served as project director at CMG (now CGI) in the Netherlands. He is co-author of Integrated Test Design and Automation and a frequent speaker at international conferences.