Language menu for mobile


Business Test Policies by Hans Buwalda

In a previous newsletter, I discussed Test Governance, the topic of organizing and managing testing activities in an organization. In this article, I want to discuss something called "business test policies." These are statements that serve as basis for the Test Governance, and describe how testing is positioned in the overall company strategy, environment and culture.

Business test policies give the corporate perspective on testing (and test automation), using explicit policy statements. An example of a business test policy is "Performance testing is a responsibility of the system development groups", or "tests and their automation are regarded as company assets and need to be managed accordingly".

These policies are not something that many companies or institutions will have developed, but it makes sense to spend some time here, since software quality is critical for business and testing is hard to organize and manage. Considering that in a typical IT organization, testing makes up about 30% of costs, it deserves the attention.

Let me give you the bad news first: to be effective, business test policies need to come from upper level management. Directions on testing, like how much it may cost, are business decisions: too much testing means too much expenditure, while too little testing introduces risks that threaten the company's revenue. This means test managers need to engage in deep discussions with upper level management about the testing objectives, which can be intimidating. Due to their experience with these sorts of discussions, external consultants can be quite helpful in these situations.

The good news: Developing good policies shouldn't take too long. Most companies I have been in already have some sense of the position of testing. For example, in a recent discussion with a major technology company about a consumer product, they told me the product must not contain any bugs that are visible to the user; another company dealing with specialized geological data was generally tolerant for bugs in dialogs and controls, as long as the data underlying data was flawless.

Policy statements need to be meaningful, not just "lip service". A statement along the lines of "testing is good, bugs are bad" is not enough. The best way to think about it is in terms of money: every statement has cost consequences, so is it important enough to justify the cost?

Thinking about testing in business terms is challenging. Testers should not try to, or be expected to, set their own goals "in a bubble" (i.e. "we should test, because it is good to test"). Unfortunately this is the case more often than not, and it leads to a lack of commitment from the rest of the organization to the testing effort. Testing costs time and money, so there should be a business reason, coming from a business manager, to test. A business manager is responsible for costs and profits: he or she is accountable for money spent on testing, but also responsible if the company loses money because a system was released without enough testing. For a tester/test manager, life is much more comfortable if there is a clear assignment from the business, i.e. what to test, and what the budget is.

This leads to the hardest part in getting business test policies: If you as a test manager want to establish them effectively, it is best to think in business terms and address business issues, without even considering testing at first. I call this a "U-turn": you step out of your testing world, engage in a business discussion, and then translate the business considerations back to testing and test automation considerations.

Here are some of the concerns that an organization can formulate business test policies for:

  • What is the significance of testing and how much can be spent on it?
  • How does testing connect to critical success factors of the company?
  • Do we have problems, and if yes, what are their causes?
  • When should testing be done in the system development life cycle?
  • Who should be involved in testing (test development, assessment, reporting) and who is responsible for testing?
  • What testing expertise is needed, and how will it be provided?
  • Is testing centralized or decentralized in the organization?
  • Are there methods and tools that should be used?
  • What degree of test automation should be used?
  • What (if any) degree of outsourcing of (1) testing, (2) development of tests and/or (3) automation should be used?

Most importantly, keep an exercise on business test policies practical. That way they can contribute to an effective and efficient test process.