WANT TO:

March 2007 - What Makes Good Testing

Rob Pirozzi, LogiGear Corporation

This article was developed from concepts in the book Global Software Test Automation: Discussion of Software Testing for Executives.

Introduction

Good testing starts with providing proper methodology, environment and the funding to carry it through. Designing good tests requires training and good project management. The test team should be involved in the requirements review process to gain a solid understanding of the product, and begin designing tests early in the development process. For test teams that rely on manual software testing, execution of the tests consumes most of their time on a project; this time commitment is often the most significant problem the development organization faces.

If the product requirements are not well-defined, or the schedule slips or becomes too tight, the development organization may cut or eliminate the time allocated for designing tests and exploratory testing may be performed immediately instead. This causes project management problems and a loss of confidence in the product of the testing process. The rest of the development organization then has no assurance of what the test team is doing, and the test team can only rely on their gut instincts for what tests they perform, since they have not gone through the necessary process of designing good tests. While gut instinct does have a role in software testing, it does not engender confidence in the product of software testing when it is used as its basis. You should run as many effective and well-designed tests that address high risk and important areas as you need, to give you the most meaningful visibility and confidence and enable you to sleep well at night.

What makes up good test designs?

Validation, the confirmation that the software indeed provides the features that were specified, is not sufficient for testing. Software testing is more than validation. Indeed, it is important to do more than validate the software, and in some cases, even more important to attempt to break the software than validate it.

For example, if you are releasing a limited-functionality demo version of your software for free distribution on the web, it is vastly more important to attempt to break the software to ensure that you cannot access the features reserved for the full version of the software than it is to confirm the existence of the features that are supposed to be there. Imagine the financial loss if you accidentally left the key functionality of your expensive software in the give-away version.

Most testers spin their wheels doing validation, but that does not really improve the quality of the software. Quality control/validation is only a subset of testing. It is a different discipline. Testing sees if the software works in the manner that the customer wants it to, and does not break even when the user makes mistakes or misuses the product. Validation checks that the software meets the requirements.

The time investment in test execution is the largest single quality cost for most development organizations today, and is well understood. Cutting this time is the highest priority for most organizations, but has typically met with poor results. For example, this has resulted in a lack of confidence in the product. While it is not expected that testing will find every bug, which it can't do, many companies have nothing more than their good wishes and hopes that the software they release is of substantive quality, won't damage their reputation, or result in a customer support nightmare. In addition, cutting the test execution time often results in poor communication due to the resulting changes in approved test plans and coverage matrices that make development teams wonder what is going on in testing. Since most of this occurs at the end of the development cycle, there is a feeling that testing is an albatross on development.

The test team should have a number of things to enable management to make better decisions regarding product quality. These include the following:

  • Effective training.
  • Continuous skills development.
  • Efficient implementation of test tools.
  • More effective communication from the test team to the product development team and management.

Conclusion

Delivering higher quality products from the test team helps to relieve management stress and fosters better decision-making. Providing the test team with the proper methodology, environment and the funding to carry through good testing and test design is essential to delivering higher quality products.

Other Articles in This Series

Other articles in this series, also based on concepts in the book Global Software Test Automation: A Discussion of Software Testing for Executives, include:
 
Back Top