Testing Computer Software Second Edition

Common Software Errors - Calculation Errors

CALCULATION ERRORS

The program calculates a number and gets the wrong result. This can happen for one of three types of reasons:

• Bad logic: There can be a typing error, like A-A instead of A+A. Or the programmer might break a complex expression into a set of simpler ones, but get the simplification wrong. Or he might use an incorrect formula, or one inapplicable to the data at hand. This third case is a design error. The code does what the programmer intendedóit's his conception of what the code should do that is wrong.
• Bad arithmetic: There might be an error in the coding of a basic function, such as addition, multiplication, or exponentiation. The error might show up whenever the function is used (2 + 2 = -5) or it might be restricted to rare special cases. In either case, any program that uses the function can fail.
• Imprecise calculation: If the program uses floating point arithmetic, it loses precision as it calculates, because of roundoff and truncation errors. After many intermediate errors, it may claim that 2 + 2 works out to -5 even though none of the steps in the program contains a logical error.

This area is huge and this section only begins to scratch its surface. For an introduction to the larger area, read Conte and deBoor (1980) and Knuth (1981). For a second source on topics in Conte and deBoor, try Carnahan, Luther & Wilkes (1969).

OUTDATED CONSTANTS

Numbers are sometimes used directly in a program. The computer might be able to connect to a maximum of 64 terminals. The length of the configuration file might be 706 bytes. The first two digits in the year are 19 (as in 1987). When these values change, the program has to change too. Often, they are changed in a few places but not everywhere. Any calculations based on the old values are now out of date, and thus wrong.

CALCULATION ERRORS

Some errors are as simple as typing a minus sign instead of a plus, or subtracting B from A instead of A from B. These usually show up easily if you exercise reasonable care in testing. If the program asks for input data, then prints a number calculated from these, do the same calculation yourself. Does your number match the computer's?

IMPOSSIBLE PARENTHESES

(A + (B + C) * (D + (A / C - B E / (B + (F + 18 / (A - F))))))

Formulas with many parentheses are hard to understand. It's easy to get one wrong, when writing the code in the first place and when trying to change it later.

WRONG ORDER OF OPERATORS

The program will evaluate an expression in a certain order, but it might be a different order than the programmer expected. For example, if ** represents exponentiation, so 5 ** 3 is 5 cubed, is 2 * 5 ** 3 equal to 1000 (10 cubed) or 250 (twice 5 cubed)?

BAD UNDERLYING FUNCTION

Commercially supplied programs and languages usually do the most basic functions, such as adding and multiplying, correctly. Of course, if your development group has written their own, these functions are as suspect as everything else. Slightly fancier functions, like exponentiation, sine, cosine, and hyperbolics, are not necessarily as trustworthy. Errors of this class may be deliberate. Some programmers use inaccurate approximation formulas because they evaluate quickly or are compact and easy to code.

OVERFLOW AND UNDERFLOW

An overflow condition occurs when the result of a numerical calculation is too large for the program to handle. For example, suppose the program stores all numbers in fixed point format, with one byte per number. It works with numbers from 0 to 255. It can't add 255 + 255 because the result is too large to fit in one byte. Overflows also occur in floating point arithmetic, when the exponent is too large.

Underflows occur only in floating point calculations. In floating point, a number is represented by a pair of values, one for the exponent, the other for a fraction. For example, 255 is 0.255 times 103. 255,000 is 0.255 times 106. The exponent changes, but the fractional part (0.255) is the same in both cases. Now, suppose the program allocates a byte for the exponent, and stores values of 0 to 255. What happens if the exponent is -1 (0.255 * 10-1 is 0.0255)? This is too small to be stored (because the smallest exponent we can store in this scheme is 0), so we have an underflow. Underflows are usually converted to 0 (0.255 * 10-1 becomes 0), without an error message. This is usually appropriate, but it can lead to computational errors:

Is 100 * .255 * 10-1 zero or 2.55?

TRUNCATION AND ROUNDOFF ERROR

Suppose the program stores only two digits per number. The number 5.19 has three digits. If the program truncates (drops) the 9, it stores 5.1. Instead, it could round 5.19 upwards to 5.2, which is much closer than 5.1.

If a programming language keeps two digits per floating point number, it works in two-digit precision. Calculations in this language would not be accurate. For example, 2.056 is about 74. However, if you round 2.05 to 2.1, the calculated value of 2.056 is nearly 86. If you truncate 2.05 to 2.0, you get 64 instead.

We don't know of any language that uses two-digit precision, but many keep only six digits in floating point calculations. Six-digit calculations are fine for simple computations but they can cause surprisingly large errors in more complicated calculations.

CONFUSION ABOUT THE REPRESENTATION OF THE DATA

The same number can be represented in many different ways; they can be confused with each other. For example, suppose that the program asks you for a number between 0 and 9. You type 1. It might store this 1 in a byte, as fixed point number between 0 and 255. There are 8 bits in this byte: the bit pattern is 0000 0001. Or it might store the ASCII code of the character you typed. The ASCII code for 1 is 49, or 0011 0001 in binary. In both cases, the number fits in one byte. It is easy to get confused later and treat something stored in ASCII format as if it were as a fixed point integer, or vice versa. Your number might also be stored in some other format, such as floating point. Again, confusion between formats is possible. Some programming languages straighten this out themselves, others issue warnings during compilation, and others just let you get the wrong answer.

INCORRECT CONVERSION FROM ONE DATA REPRESENTATION TO ANOTHER

The program asks for a number between 0 and 9. You type 1. This is a character, and the program receives the ASCII code 49. To convert your input to a number, it should subtract 48 from the code value. It subtracts 49 instead. You entered 1, but the program will treat your response as 0. Whenever a program does its own conversion from one data representation to another, it has plenty of opportunity to go wrong. ASCII to Integer is only one example of a conversion. Conversions are common between ASCII, floating point, integer, character (string), etc.

WRONG FORMULA

Some programs use complicated formulas. It's easy to miscopy one, to read the wrong one from the book, or to make an error when deriving one.

INCORRECT APPROXIMATION

Many formulas for approximating or estimating certain values were developed long before computers. They're great formulas in the sense that they don't require much computation, but lousy otherwise. Unfortunately, they are also traditional. They will keep appearing in text books and programs for years. As a common example, if you are graphing a set of data and want to fit a curve to them that has the form Y = a Xb, it is traditional to take logarithms of everything in sight, estimating a and b by fitting a line to the new function, log Y = log a + b log X. This is easy to program, quick to run, and inaccurate. When you return from logarithms and plot a Xb, the curve fits data on the left of the figure (small values of X) better than data toward the right.

Many programs use bad approximation methods and other incorrect mathematical procedures. They might print impressive output, but it is wrong output. You can't test for these types of problems unless you understand a fair bit of the mathematics yourself. If you are testing a statistical package or other mathematical package, it is essential that you or another tester have a detailed understanding of the functions being programmed.