Testing Computer Software Second Edition

Common Software Errors - Load Conditions

LOAD CONDITIONS

Programs misbehave when overloaded. A program may fail when working under high volume (lots of work over a long period) or under stress (maximum amount of work all at once). It may fail when it runs out of memory, printers or other "resources." It may fail because it's required to do too much in too little time. All programs have limits. The issues are whether a program can meet its stated limits and how horribly it fails when those limits are exceeded.

Also, some programs create their own load problems, or, in multi-processing situations, make problems for others. They hog computer time or resources or create unnecessary extra work to such an extent that other processes (or themselves later) can't do their tasks.

REQUIRED RESOURCE NOT AVAILABLE

The program tries to use a new device or store more data in memory, but can't. tests run for the program's handling of each of the conditions below, and similar tests for any other device you use (plotters, telephones, etc.). The following conditions should be self-explanatory:

• Full disk
• Full disk directory
• Full memory area
• Full print queue
• Full message queue
• Full stack
• Disk not in drive
• Disk drive out of service
• No disk drive
• Printer off line
• Printer out of paper
• Printer out of ribbon
• No printer
• Extended memory not present

DOESN'T RETURN A RESOURCE

Systems may run out of resources because one or a few processes hog them all. Programmers are good at making sure that their programs get the resources they need. They are not as conscientious about returning resources they no longer need. Since the program won't crash (usually) if it hangs on to the printer or a memory buffer for too long, this type of problem seems less urgent.

The following sections are examples to consider when designing test cases.

Doesn't indicate that it's done with a device

For example, a process uses the printer. All other processes have to wait until this one signals that it's done. The process fails to send that signal, and so prevents others from using an unused device.

Doesn't erase old files from mass storage The program doesn't erase outdated backups and internal-use temporary files. There are limits on how much erasure should be done automatically, but the process doesn't get rid of files that obviously should go.

Doesn't return unused memory

In multi-processing systems, a memory management process can assign segments of memory to processes on a temporary basis (such as data and message buffers). The process is supposed to signal the memory manager when done with a segment. The manager takes back control of the segment and assigns it for use by other processes as needed. Failure to return buffers, especially message buffers, is extremely common.

Wastes computer time

The process checks for events that are no longer possible or does other things that used to be necessary but now aren't.

NO AVAILABLE LARGE MEMORY AREAS

In a message passing, multi-processing system, a pool of memory is available to be assigned as message buffers to any process. Some buffers are large, others just a few bytes. A large block of memory might be divided into many small buffers. What happens when the program needs a larger block of memory than any of these small ones?

Some memory management programs don't attempt to merge used buffers into the memory pool. Instead, they reuse these old buffers whenever a buffer their size is needed. As a result, there are few large areas of memory in the common pool.

INPUT BUFFER OR QUEUE NOT DEEP ENOUGH

The process loses keystrokes, messages, or other data because too much comes at once and there's nowhere to put it all. When a process receives more individual data items (like keystrokes) than it can immediately handle, it usually stores the extras in an input buffer, reading them from the buffer and dealing with them when it has time. Similarly, it might store packets of information (like messages) in a queue, getting to them one at a time.

If the process' input buffer is 10 characters deep, what happens if you type 11 (or more) characters quickly? Does it signal that the buffer is full? What if the device sending input is a computer, connected to a modem? Does the program tell the sending program to stop for a while?

If the process' message queue is 256 messages deep, what happens when 256 messages are waiting, one is being processed, and the 257th arrives? Is it discarded or is it returned to the sender with an error code signaling that the message queue is full? If a few messages are discarded without notification, what is the most consequential message discarded? What important information has the receiving process lost that the sending process will assume was received?

DOESN'T CLEAR ITEMS FROM QUEUE, BUFFER, OR STACK

Suppose the program receives messages, puts them in a queue, and reads them from the queue when it has time. It can store up to 256 messages in the queue. When it reads a message, the process should remove it from the queue, making room for a new one. However, programmers may forget to remove debugging messages, so the queue fills as soon as the program tries to use the queue for the 257th time.

In more subtle cases (similarly for failures to return buffers), messages are usually but not always removed from the queue. In one special case, the program doesn't discard old messages. It fails on the 257th time that this bug is triggered. It's because of these kinds of errors that you should occasionally test a program for a long time without rebooting it. Make sure that a minor problem, invisible over short periods, doesn't eventually devastate the system. A "long time" is defined in terms of your customers' needs. They will restart a word processor much more often than a telephone system. You might test the word processor for hours before rebooting, the telephones for weeks or months.

LOST MESSAGES

The operating system might lose some messages (presto, vanish). Or the receiving process might, if enough messages arrive at once. If a process is supposed to be able to handle a queue of 256 messages, what happens to the message it's working on, the message at the start of the queue, and the one at the end, when the 256th message arrives? What happens with the 257th, 258th, and 259th messages? Are they returned to the sending process with a "send-me-again-later" notification or are they just thrown away? Does the sending process need to know that the process it sent a message to was too busy to read it?

PERFORMANCE COSTS

When the workload is high, everything slows down. Larger arrays have to be searched, more users or processes have to be served, etc. A program that must respond within a certain time or process so many events per second might fail because it's running under too busy a multiprocessing system or because it's trying to juggle too many inputs itself. Other programs, expecting that this one will respond within a short interval, might also fail if this one responds too slowly.

RACE CONDITION WINDOWS EXPAND

As performance gets worse, race conditions become more likely. In the classic race, two events can happen. One almost always precedes the other but sometimes the second will (or will appear to) precede the first by a tiny bit. As you slow the system down, it may take the computer longer to generate or detect the first event. If the second event is an input (keystroke or modem input), the person or machine that generates it may not be affected by the increased load. The second event will happen as quickly as usual even though the processing of the first has slowed. Thus there is more time for the second event to appear to beat the first.

DOESN'T ABBREVIATE UNDER LOAD

Some processes make lots of output. Formatting all this information and sending it to the printer or screen takes lots of computer time. When the computer is operating under heavy load, an output-intensive process should try to send out less. It might express error messages more tersely, or abbreviate system log messages to short codes or send them to a buffer or disk file to be printed later. Only urgent messages should go to the printer or screen immediately.

Use your judgment before criticizing a program that doesn't abbreviate or run at lower priority. A word processing program that is about to print the agenda for a board meeting that starts in three minutes should neither delete items from the agenda nor wait till tomorrow before printing it.

DOESN'T RECOGNIZE THAT ANOTHER PROCESS ABBREVIATES OUTPUT UNDER LOAD

Imagine a multi-user system in which all programs log failures and other "interesting" events to the system operator's console. Usually, the log messages include a few numbers plus English-language descriptions. Under heavy load, the messages are abbreviated to short numeric codes. No one can understand the codes without looking them up in a book, but at least the messages themselves don't further slow down the system.

Now suppose these logs are stored on disk. At the end of every day (week, month), a maintenance program reads the files and analyzes the system's failures, perhaps running hardware diagnostics in response to some of the messages. This maintenance program will have to be able to cope with the short code abbreviation system, or it will fail whenever it has to read a message that was saved when the system was under heavy load. A dismaying number of analysis programs do not know how to deal with abbreviated output.

LOW PRIORITY TASKS NOT PUT OFF

Under heavy load, any task that doesn't have to be done immediately should be postponed. In a multi-processing system, programs or people are assigned priorities. Those with higher priorities should get a higher share of available machine time than those with lower priorities.

LOW PRIORITY TASKS NEVER DONE

You can put off changing the oil in your car for a while, but eventually it must be done or else. Many low priority tasks are like this: you don't have to do them right away, but they must be done eventually. High priority tasks cannot be allowed to use all of the computer's time under prolonged periods of heavy load. Some time must be given to lower priority tasks.