Key Principles of Test Design

Regardless of the method you choose, simply spending some time thinking about good test design before writing the first test case will have a very high payback down the line, both in the quality and the efficiency of the tests.

Test design is the single biggest contributor to success in software testing and its also a major factor for success in test automation. This is not that intuitive. Like many others, I initially thought that successful automation is an issue of good programming or even “buying the right tool”. That test design turns out to be a main driver for automation success is something that I had to learn over the years, often the hard way.

What I have found is that there are three main goals that need to be achieved in test design. I like to characterize them as the “Three Holy Grails of Test Design”, a metaphor based on the stories of King Arthur and the Round Table. Each of the three goals is hard to reach, just like it was hard for the knights of King Arthur to find the Holy Grail. This article will introduce the three “grails” to look for in test design.

The terminology in this article and is based on Action-Based Testing (ABT), LogiGear’s method for testing and test automation. You can read more about the ABT methodology on the LogiGear web site.

The Three Goals for Test Design

The three most important goals for test design are:

  • Effective breakdown of the tests

The first step is to breakdown the tests into manageable pieces, which in ABT we call “test modules”. At this point in the process we are not yet describing test cases; we simply identify the “chapters” into which test cases will fall. A break down is good if each of the resulting test modules has a clearly defined and well-focused scope, which is differentiated from the other modules. The scope of a test module subsequently determines what its test cases should look like.

  • Right approach per test module

Once the break down is done each individual test module becomes a mini-project. Based on the scope of a test module we need to determine what approach to take to develop the test module. By approach I mean the choice of testing techniques used to build the test cases (like boundary analysis, decision tables, etc.), and who should get involved to create and/or assess the tests. For example, a test module aimed at testing the premium calculation of insurance policies might need the involvement of an actuarial department.

  • Right level of test specification

This third goal is where you can win or lose most of the maintainability of automated tests. When creating a test case try to specify those, and only those, high-level details that are relevant for the test. For example, from the end-user perspective “login” or “change customer phone number” is one action; it is not necessary to specify any low-level details such as clicks and inputs. These low-level details should be “hidden” at this time in separate, reusable automation functions common to all tests. This makes a test more concise and readable, but most of all it helps maintain the test since low-level details left out will not have to be changed one-by-one in every single test if the underlying system undergoes changes. The low-level details can then be re-specified (or have their automation revised) only once and reused many times in all tests. In ABT this third principle is visible in the “level” of the actions to be used in a test module. For example, in an insurance company database, we would write tests using only “high-level” actions like “create policy” and “check premium”, while in a test of a dialog you could use a “low level” action like “click” to see if you can click the OK button.

Conclusion

Regardless of the method you choose, simply spending some time thinking about good test design before writing the first test case will have a very high payback down the line, both in the quality and the efficiency of the tests.

 

Hans Buwalda

Hans leads LogiGear’s research and development of test automation solutions, and the delivery of advanced test automation consulting and engineering services. He is a pioneer of the keyword approach for software testing organizations, and he assists clients in strategic implementation of the Action Based Testing™ method throughout their testing organizations.

Hans is also the original architect of LogiGear’s TestArchitect™, the modular keyword-driven toolset for software test design, automation and management. Hans is an internationally recognized expert on test automation, test development and testing technology management. He is coauthor of Integrated Test Design and Automation (Addison Wesley, 2001), and speaks frequently at international testing conferences.

Hans holds a Master of Science in Computer Science from Free University, Amsterdam.

Hans Buwalda
Hans Buwalda, CTO of LogiGear, is a pioneer of the Action Based and Soap Opera methodologies of testing and automation, and lead developer of TestArchitect, LogiGear’s keyword-based toolset for software test design, automation and management. He is co-author of Integrated Test Design and Automation, and a frequent speaker at test conferences.

The Related Post

This article was developed from concepts in the book Global Software Test Automation: Discussion of Software Testing for Executives. Introduction Many look upon Software Testing as a cost. While it is true that Software Testing does cost money, in many cases significant amounts of money, it is also an activity that helps an organization to ...
Think you’re up for a challenge? Print this word search out! See if you can find all the words and learn a few new software testing terms in the process. To see how you’ve done, check your answers in the answer key below. *You can check the answer key here.
LogiGear Magazine – February 2014 – Test Methods and Strategies
“Combinatorial testing can detect hard-to-find software faults more efficiently than manual test case selection methods.” Developers of large data-intensive software often notice an interesting—though not surprising—phenomenon: When usage of an application jumps dramatically, components that have operated for months without trouble suddenly develop previously undetected errors. For example, newly added customers may have account records ...
Please note: This article was adapted from a blog posting in Karen N. Johnson’s blog on July 24, 2007. Introduction The password field is one data entry field that needs special attention when testing an application. The password field can be important (since accessing someone’s account can start a security leak), testers should spend more ...
Introduction Keyword-driven methodologies like Action Based Testing (ABT) are usually considered to be an Automation technique. They are commonly positioned as an advanced and practical alternative to other techniques like to “record & playback” or “scripting”.
There are many ways to approach test design. These approaches range from checklists to very precise algorithms in which test conditions are combined to achieve the most efficiency in testing. There are situations, such as in testing mobile applications, complex systems and cyber security, where tests need to be creative, cover a lot of functionality, ...
From cross-device testing, to regression testing, to load testing, to data-driven testing, check out the types of testing that are suitable for Test Automation. Scene: Interior QA Department. Engineering is preparing for a final product launch with a deadline that is 12 weeks away. In 6 weeks, there will be a 1 week quality gate, ...
In software testing, we need to devise an approach that features a gradual progression from the simplest criteria of testing to more sophisticated criteria. We do this via many planned and structured steps, each of which brings incremental benefits to the project as a whole. By this means, as a tester masters each skill or area ...
VISTACON 2010 – Keynote: The future of testing THE FUTURE OF TESTING BJ Rollison – Test Architect at Microsoft VISTACON 2010 – Keynote   BJ Rollison, Software Test Architect for Microsoft. Mr. Rollison started working for Microsoft in 1994, becoming one of the leading experts of test architecture and execution at Microsoft. He also teaches ...
I’ve been intending to write a book review of How We Test Software At Microsoft, by Alan Page, Ken Johnston, and Bj Rollison, but for whatever reason I just never found the time, until now. In general, I like this book a lot. It’s a nice blend of the tactical and the strategic, of the ...
This is an adaptation of a presentation entitled Software Testing 3.0 given by Hung Nguyen, LogiGear CEO, President, and Founder. The presentation was given as the keynote at the Spring 2007 STPCON conference in San Mateo, California. Watch for an upcoming whitepaper based on this topic. Introduction This first article of this two article series ...

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe