Understanding Quality Cost

This article was developed from concepts in the book Global Software Test Automation: Discussion of Software Testing for Executives.

Quality cost is the sum of all costs a company invests into the release of a quality product. When developing a software product, there are 4 types of quality costs: prevention costs, appraisal costs, internal failure costs, and external failure costs.

  1. Prevention costs represent everything a company spends to prevent software errors, documentation errors, and other product-related errors. These include requirements and usability analysis, for example. Dollars spent on prevention costs are the most effective quality dollars, because preventing errors from getting into the product is much cheaper than fixing errors later. If there is an error in a requirement or the intended usability, and money is spent on developing the software to the erroneous requirement, the costs of identifying the error, determining how to fix it, and then developing new code to correct it will arise later.
  2. Appraisal costs include the money spent on the actual testing activity. Any and all activities associated with searching for errors in the software and associated product materials falls into this category. This includes all testing by the developers themselves, by an internal test team, and by an outsourced software test organization. This also includes all associated hardware, software, labor, and other costs. Once a product is in the coding phases, the goal is to do the most effective appraisal job, so that internal failure work is streamlined, well-managed, and prevents skyrocketing external failure costs.
  3. Internal failure costs are the costs of coping with errors discovered during development and testing. These are bugs found before the product is released. As we mentioned previously, the further in the development process the errors are discovered, the more costly they are to fix. So, the later the errors are discovered, the higher their associated internal failure costs will be.
  4. External failure costs are the costs of coping with errors discovered after the product is released. These are typically errors found by your customers. These costs can be much higher than internal failure costs, because the stakes are much higher. These costs include post-release customer and technical support. Errors at this stage can also be costly in terms of your company’s reputation and may lead to lost customers.

The following table shows examples of costs that fall into each of the four categories of quality costs.

PreventionAppraisal
Staff training
Requirements analysis
Early prototyping
Fault tolerant design
Defensive programming
Usability analysis
Clear specification
Accurate internal documentation
Pre-purchase evaluation of the
reliability of development tools
Design reviews
Code inspection
Glass box testing
Black box testing
Beta testing
Test Automation
Usability testing
Pre-release out-of-box testing
by customer service staff
Internal FailureExternal Failure
Bug fixes
Regression testing
Wasted in-house user time
Wasted tester time
Wasted writer time
Wasted marketer time
Wasted advertisements
Direct cost of late shipment
Opportunity cost of late shipment
Technical support calls
Answer books (for support)
Investigating complaints
Refunds and recalls
Interim bug fix releases
Shipping product updates
Warranty, liability costs
PR to soften bad reviews
Lost sales
Lost customer goodwill
Supporting multiple versions in the
field
Reseller discounts to keep them
selling the product

The Four Categories of Quality Costs

All the costs mentioned above can be effectively reduced through smarter test efforts that include a high degree of Test Automation. Test Automation when done right leads to greater test coverage, resulting in higher-quality products. Higher-quality products require less technical support, fewer patches, and lead to greater customer satisfaction. Smarter Automated Testing also speeds up the release process and incrementally reduces the manual test costs. But most of all, more test coverage gives you and your customers more confidence in your product. You will feel more comfortable knowing that there are not bugs lurking in your software that have not been exposed yet because of insufficient test coverage. You will also not have to scramble at the last minute (typically, on the first day of that much-needed vacation) to deal with a problem and fix it to your customer’s satisfaction in a rush.

The solution to quality cost problems is to get a better understanding of your investment in product quality and manage your costs better. The first place most organizations look for a better understanding is in the highest cost area: the software test effort, or lack thereof. For example, if you do not test at all, your testing or appraisal cost is low. You will ship on time, but your external failure costs will skyrocket. Your prevention and appraisal costs will result in finding errors that can be corrected while they are still internal failures, where they are cheaper to deal with.

The goal of understanding quality costs is to analyze where you spend your time and money to get the most bang for the buck. It is well known that it is faster and cheaper to find and fix a bug during unit testing done by developers early in the development cycle. Should we then spend most of our time/budget on unit testing? No. There are many limitations to unit testing. Unit testing is not capable of finding many varieties of bugs, including graphical user interface (GUI) bugs, usability problems, End-to-End bugs, and configuration bugs. For most organizations, getting a better unit test effort will help you release a better product sooner. It is not a replacement for the test effort done by skilled software testers, but it may reduce the time that test effort takes. Understanding quality costs will hopefully help you shift some of your test effort to the most cost-effective places.

In the following, the total quality cost is shown in the upper bathtub-shaped curve. On the bottom axis is the quality of performance, ranging from totally defective to zero defects. On the left axis is the cost per good unit of product. You can see that with highly defective software, your prevention and appraisal costs are very low, but your failure costs are very high, yielding a high total quality cost. With zero defect software, likewise, your failure costs are very low, but your prevention and appraisal costs are very high. To optimize your total quality costs, you want to be between these extremes, at the bottom of the bathtub curve.

Theoretical Model of Optimum Quality Cost

This offers 2 challenges. First, a sufficiently sophisticated accounting system allowing a typical mid-sized company to track the total cost of quality has yet to be developed. To optimize total quality cost, you need to have the appropriate categories in your accounting system and keep track of the related costs. Second, you need to be able to track your external quality costs. You may not even have enough information from customers on why the software is not working for them. How are you going to know what to book into your accounting system for external failure costs? The point here is that while capturing this data is difficult and expensive, you know that the benefit is reducing your overall cost of quality. You need to determine if the benefits of tracking your total quality cost will give you enough of a return on investment to make setting up the appropriate accounting system and paying for the implementation of the program worthwhile.

Conclusion

Taking the time to better understand you quality costs will help you to much more cost effectively deliver a quality software product. You will be able to optimize the various costs to achieve the best quality achievable at a more reasonable price.

[1] J. M. Juran, Quality Control Handbook, 3rd Edition. New York: McGraw Hill, 1974.

Hung Nguyen

Hung Nguyen co-founded LogiGear in 1994, and is responsible for the company’s strategic direction and executive business management. His passion and relentless focus on execution and results has been the driver for the company’s innovative approach to software testing, test automation, testing tool solutions and testing education programs.

Hung is co-author of the top-selling book in the software testing field, “Testing Computer Software,” (Wiley, 2nd ed. 1993) and other publications including, “Testing Applications on the Web,” (Wiley, 1st ed. 2001, 2nd ed. 2003), and “Global Software Test Automation,” (HappyAbout Publishing, 2006). His experience prior to LogiGear includes leadership roles in software development, quality, product and business management at Spinnaker, PowerUp, Electronic Arts and Palm Computing.

Hung holds a Bachelor of Science in Quality Assurance from Cogswell Polytechnical College, and completed a Stanford Graduate School of Business Executive Program.

Rob Pirozzi

Over 20 years of sales, marketing, management, and technology experience in high technology with exposure to industries including financial services, healthcare, higher education, government, and manufacturing; demonstrating a strong track record of success. Proven ability to build and maintain strong relationships, contribute to target organization success, and deliver results. Website: http://www.robpirozzi.com/

Hung Q. Nguyen
Hung Nguyen co-founded LogiGear in 1994, and is responsible for the company’s strategic direction and executive business management. His passion and relentless focus on execution and results has been the driver for the company’s innovative approach to software testing, test automation, testing tool solutions and testing education programs.
Hung is co-author of the top-selling book in the software testing field, “Testing Computer Software,” (Wiley, 2nd ed. 1993) and other publications including, “Testing Applications on the Web,” (Wiley, 1st ed. 2001, 2nd ed. 2003), and “Global Software Test Automation,” (HappyAbout Publishing, 2006). His experience prior to LogiGear includes leadership roles in software development, quality, product and business management at Spinnaker, PowerUp, Electronic Arts and Palm Computing.
Hung holds a Bachelor of Science in Quality Assurance from Cogswell Polytechnical College, and completed a Stanford Graduate School of Business Executive Program.
Hung Q. Nguyen on Linkedin

The Related Post

Having developed software for nearly fifteen years, I remember the dark days before testing was all the rage and the large number of bugs that had to be arduously found and fixed manually. The next step was nervously releasing the code without the safety net of a test bed and having no idea if one ...
“Combinatorial testing can detect hard-to-find software faults more efficiently than manual test case selection methods.” Developers of large data-intensive software often notice an interesting—though not surprising—phenomenon: When usage of an application jumps dramatically, components that have operated for months without trouble suddenly develop previously undetected errors. For example, newly added customers may have account records ...
Introduction Software Testing 3.0 is a strategic end-to-end framework for change based upon a strategy to drive testing activities, tool selection, and people development that finally delivers on the promise of software testing. For more details on the evolution of software testing and Software Testing 3.0 see: Software Testing 3.0: Delivering on the Promise of ...
It’s a bird! It’s a plane! It’s a software defect of epic proportions.
PWAs have the ability to transform the way people experience the web. There are a few things we can agree we have seen happen. The first being that we figured out the digital market from an application type perspective. Secondly, we have seen the rise of mobile, and lastly, the incredible transformation of web to ...
One of the most common challenges faced by business leaders is the lack of visibility into QA activities. QA leaders have a tough time communicating the impact, value, and ROI of testing to the executives in a way that they can understand. Traditional reporting practices often fail to paint the full picture and do not ...
I’ve been reviewing a lot of test plans recently. As I review them, I’ve compiled this list of things I look for in a well written test plan document. Here’s a brain dump of things I check for, in no particular order, of course, and it is by no means a complete list. That said, if you ...
This article was developed from concepts in the book Global Software Test Automation: Discussion of Software Testing for Executives. Article Synopsis There are many misconceptions about Software Testing. This article deals with the 5 most common misconceptions about how Software Testing differs from other testing. Five Common Misconceptions Some of the most common misconceptions about ...
Introduction Software Testing 3.0 is a strategic end-to-end framework for change based upon a strategy to drive testing activities, tool selection, and people development that finally delivers on the promise of Software Testing. For more details on the evolution of Software Testing and Software Testing 3.0 see: The Early Evolution of Software Testing Software Testing ...
Reducing the pester of duplications in bug reporting. Both software Developers and Testers need to be able to clearly identify any ‘Bug’, via the ‘Title’ used for the ‘Bug Report’.
This article was adapted from a presentation titled “How to Optimize Your Web Testing Strategy” to be presented by Hung Q. Nguyen, CEO and founder of LogiGear Corporation, at the Software Test & Performance Conference 2006 at the Hyatt Regency Cambridge, Massachusetts (November 7 – 9, 2006). Click here to jump to more information on ...
This article was developed from concepts in the book Global Software Test Automation: Discussion of Software Testing for Executives. Introduction Metrics are the means by which the software quality can be measured; they give you confidence in the product. You may consider these product management indicators, which can be either quantitative or qualitative. They are ...

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe