Testing Computer Software Second Edition

Common Software Errors - Boundary-Related Errors

BOUNDARY-RELATED ERRORS

A boundary describes a change-point for a program. The program is supposed to work one way for anything on one side of the boundary. It does something different for anything on the other side. The classic "things" on opposite sides of boundaries are data values. There are three standard boundary bugs:

• Mishandling of the boundary case: If a program adds any two numbers that are less than 100, and rejects any greater than 100, what does it do when you enter exactly 100? What is it supposed to do?
• Wrong boundary: The specification says the program should add any two numbers less than 100 but it rejects anything greater than 95.
• Mishandling of cases outside the boundary: Values on one side of the boundary are impossible, unlikely, unacceptable, unwanted. No code was written for them. Does the program successfully reject values greater than 100 or does it crash when it gets one?

We treat the concept of boundaries more broadly. Boundaries describe a way of thinking about a program and its behavior around its limits. There are many types of limits: largest, oldest, latest, longest, most recent, first time, etc. The same types of bugs can happen with any of them so why not think of them in the same terms?

NUMERIC BOUNDARIES

Some numeric boundaries are arbitrary (bigger or less than 100) while others represent natural limits. A triangle has exactly three sides (not more, not less). Its angles sum to 180 degrees. A byte can store a (nonnegative) number between 0 and 255. If a character is a letter, its ASCII code is between 65 and 90 (capitals) or between 97 and 122 (lowercase).

EQUALITY AS A BOUNDARY

Every element in a list might be the same. Every element might be different. What happens if you try to sort either list? If the list is made of numbers, what happens if you try to compute their mean, standard deviation, coefficient of symmetry or kurtosis? (All four are summary statistics. The last two would either compute to 0 or cause a divide by 0 error depending on the calculation algorithm.)

BOUNDARIES ON NUMEROSITY

An input string can be up to 80 characters long? What if you type 79, 80 or 81? What does the program receiving your input do in each case? Can a list have one element? No elements? What is the standard deviation of a one-number list of numbers? (Answer: undefined or zero)

BOUNDARIES IN SPACE

For example, if a graphing program draws a graph and a box around it, what to do with a dot that should properly be displayed outside the box?

BOUNDARIES IN TIME

Suppose the program displays a prompt, waits 60 seconds for you to respond, then displays a menu if you haven't typed anything. What happens if you start typing just as it's starting to display the menu?

Suppose you have 30 seconds to answer a ringing telephone. After that, your phone stops ringing and the call forwards to the operator. Do you lose the call if you reach it at the 30th second? What if you reach it after the 30th second but before the operator has answered it?

Suppose you press the Space Bar while the computer's still loading the program from the disk. What happens? Is sent to the operating system (which is loading the program), saved for the program being loaded, or is this just so unexpected that it crashes the computer?

BOUNDARIES IN LOOPS

Here's an example of a loop:

10 IF COUNT_VARIABLE is less than 45
  THEN PRINT "This is a loop"
     SET COUNT_VARIABLE to COUNT_VARIABLE + 1
     GOTO 10
  ELSE quit

The program keeps printing and adding 1 to COUNT_VARIABLE until the counter finally reaches 45. Then the program quits. 45 bounds the loop. Loops can have lower as well as upper bounds (IF COUNT_VARIABLE is less than 45 and greater than 10). Beizer (1990) discusses tests of loop boundaries.

BOUNDARIES IN MEMORY

What are the largest and smallest amounts of memory that this program can cope with? (Yes, a few programs do crash if you give them access to too much memory.) Are data split across pages or segments of memory? Is the first or last byte of a segment lost or misread? (By the way, is that first byte numbered 0 or 1?) Are some of the data in RAM and some on disk, in virtual memory format? Suppose the program reads a value from RAM then a value from virtual memory, then the next value from what used to be in RAM, then back to what used to be (still is? is again?) virtual memory, etc. How seriously will this back-and-forth affect performance?

BOUNDARIES WITHIN DATA STRUCTURES

Suppose the program keeps data in a record structure. Each record holds a person's name, followed by their employee number and salary. Then comes the next person's record (name, number, salary), etc. If it retrieves them from disk, does the program read the first record correctly? The last record? How does the program mark the end of a record or the beginning of the next? Does everything fit in this format? What if you have two employee numbers?

HARDWARE-RELATED BOUNDARIES

If a mainframe can handle up to 100 terminals, what happens when you plug in the 99th, 100th, and 101st? What if you get 100 people to log on at the same time? What happens when the disk is full? If a directory can hold 128 files, what happens when you try to save the 127th, 128th, and 129th? If your printer has a large input buffer, what happens if your program fills it but has more data to send? What happens when the printer runs out of paper or the ribbon runs out?

INVISIBLE BOUNDARIES

Not all boundary conditions are visible from the outside. For example, a subroutine might approximate the value of a function, using one approximation formula when the function argument is less than 100 and a different approximation if the argument is 100 or greater. The first formula might be incalculable (e.g., divide by zero) when the function argument is 100 and its values might make no sense when arguments are greater than 100. 100 is clearly a boundary but you might never realize it.

CALCULATION ERRORS

The program calculates a number and gets the wrong result. This can happen for one of three types of reasons:

• Bad logic: There can be a typing error, like A-A instead of A+A. Or the programmer might break a complex expression into a set of simpler ones, but get the simplification wrong. Or he might use an incorrect formula, or one inapplicable to the data at hand. This third case is a design error. The code does what the programmer intendedóit's his conception of what the code should do that is wrong.
• Bad arithmetic: There might be an error in the coding of a basic function, such as addition, multiplication, or exponentiation. The error might show up whenever the function is used (2 + 2 = -5) or it might be restricted to rare special cases. In either case, any program that uses the function can fail.
• Imprecise calculation: If the program uses floating point arithmetic, it loses precision as it calculates, because of roundoff and truncation errors. After many intermediate errors, it may claim that 2 + 2 works out to -5 even though none of the steps in the program contains a logical error.

This area is huge and this section only begins to scratch its surface. For an introduction to the larger area, read Conte and deBoor (1980) and Knuth (1981). For a second source on topics in Conte and deBoor, try Carnahan, Luther & Wilkes (1969).

OUTDATED CONSTANTS

Numbers are sometimes used directly in a program. The computer might be able to connect to a maximum of 64 terminals. The length of the configuration file might be 706 bytes. The first two digits in the year are 19 (as in 1987). When these values change, the program has to change too. Often, they are changed in a few places but not everywhere. Any calculations based on the old values are now out of date, and thus wrong.