|
Testing Computer Software Second Edition
Common Software Errors
|
This is the appendix from the best-selling book
Testing Computer Software, 2nd ed.
Copyright © 1988 by Cem Kaner
Copyright © 1993 by Cem Kaner, Jack Falk, Hung Quoc Nguyen
This is part 1 of 13.
[ 1 ]
[ 2 ]
[ 3 ]
[ 4 ]
[ 5 ]
[ 6 ]
[ 7 ]
[ 8 ]
[ 9 ]
[ 10 ]
[ 11 ]
[ 12 ]
[ 13 ]
|
This Appendix describes over 400 bugs. The descriptions are short, including only what we considered the most interesting information. It's worthwhile reading this list, even though you may find it boring. On first reading it provides a frame of reference, details, and background about problems you should look for while testing. Its greater value is as an organized list of program problems for future reference. A good list, built up with time and experience, can be a powerful tool.
A set of test materials is nothing more than a list of possible problems in a program and a set of procedures for determining whether the problems are actually there or not.
Generating reference lists of bugs on the fly is risky and inefficient. It takes too long to dredge up old insights, and you're too likely to get stuck on one or a few themes. We prefer to check a comprehensive "standard" list to get further ideas for testing a program. This Appendix is an example of a standard list.
Whenever you test a new program, you'll think of new entries for this list. Add them to it. We recommend that you enter this list onto your computer and manage it with an outline processor.
Here are examples of ways you should use this list:
- Evaluate test materials developed for you by someone else.
When you contract out the development of test materials you're likely to get back many thick binders of test cases. They look impressive, but they're incomplete. The author has blind spots, which you have to find. For example, in stacks of test materials that we've seen developed by contract testing agencies, we've yet to see an explicit race condition test. Unfortunately, it's easier to see what an author had in mind than to step back and see what's missing.
We recommend evaluating the coverage of test materials against a checklist. Include a few examples of each class of error that you can imagine existing in the program. It doesn't matter whether the program has these problems - it only matters that they could be there. Once the list is complete, check the test materials to see which problems they would expose. If these tests wouldn't expose some errors in your list, look for tests for similar problems. This can quickly expose large classes of missing tests.
It's easy to make a checklist from a large error list like the one in this Appendix. Just look for classes of errors that are relevant to your testing project and find a few good examples of each. Add any other problems that you think of that seem important for the program under test.
- Developing your own tests.
Write your own list of errors that you think the program could have. Then look here for more ideas. Consider each problem in this Appendix: if you're sure it's irrelevant to the program, forget it. Otherwise write a test case (or a few) to see if the program has the problem. When in doubt, ask the programmer whether a given type of error is possible in the program and
how to test for it.
We also recommend this list for testing "on the fly." If you don't have time to plan and document a set of tests for a program, test from a list of candidate problems.
- Irreproducible bugs.
A bug is hard to replicate because you don't know what triggered the failure, what the
program did or displayed, or what could cause such a problem. You give up because you run
out of hypotheses.
Use this Appendix before giving up.
Try to match reported symptoms with problems listed in this Appendix. Be creative: the
matches won't be perfect because the reports are incomplete and incorrect. (If they weren't,
you could reproduce the bug.) Even so, the list can help you break out of your current line of
thinking. If you didn't need to refocus on other possibilities, you wouldn't be so close to
giving up.
- Unexpected Bugs.
When you discover a bug accidentally, or when one surfaces in a shipping product, look for
others like it. You probably missed running a group of tests to detect the lot. This section can
help you generate a list of possible related problems and tests.
OUTLINE OF COMMON SOFTWARE ERRORS
Other Articles by Cem Kaner
Other Articles by Hung Nguyen
|
