Four Drivers for Delivering Software Test Automation Success

Test automation provides great benefits to the software testing process and improves the quality of the results. It improves reliability while minimizing variability in the results, speeds up the process, increases test coverage, and ultimately can provide greater confidence in the quality of the software being tested. However, automation is not a silver bullet. It also brings some problems. The solution for test automation is to first define the test methodology, then choose the right enabling technology to help you implement the methodology. The chosen methodology should provide measurable improvements to the following four success drivers:

  • Visibility
  • Re-usability
  • Scalability
  • Maintainability

The four drivers above have a direct impact on three factors, manageability, productivity and cost efficiency, which ultimately lead to the tangible benefits that we are looking for a successful automation program to deliver.

Consider the following Figure 1. This diagram shows the results of a good test automation methodology. Good automation provides optimum productivity to the software testing effort. Hence, it leads to higher quality of the software releases. Test automation visibility provides measurability and control over the software development process, which keeps it manageable. With good visibility established, you can make effective management decisions about if, when, and how to do training and auditing to address the quality of tests. Reusability and scalability of test automation improves test productivity. However, productivity should be defined by (1) the quantity of tests (driven by reusability and scalability), and (2) the quality of tests (visibility into what the tests are actually doing helps improve the tests qualitatively). Quality and quantity are two different things. When test automation is reusable and scalable, the issue of quantity is resolved. When test automation is highly maintainable, the cost of ownership is minimized. Building test automation in a way that is more maintainable will lower the total cost of ownership of the test assets, making the overall testing effort more cost effective.

Figure 1. Outflows of Test Automation.

The quality of tests is mostly affected by the training and skills of the test staff. Automating a bad test does not improve its quality; it just makes it run faster. Good test design is a critical and often overlooked aspect of test automation. Test automation visibility by itself does not provide high quality tests. It merely enables us to have a view into how well the test designers are trained. Addressing the training issues will help in addressing test case quality issues.


Securing and developing competency is a secret to automation success.


Visibility, reusability, scalability, and maintainability lead to productivity and are the drivers for following benefits:

  • Improved time-to-market.
  • Improved quality of releases.
  • Improved predictability.
  • Improved Test/QA communication.
  • Higher test coverage.
  • Lower testing costs.
  • Earlier detection of bugs.
  • Lower technical support costs.
  • More effective use of testing resources.
  • Improved customer confidence and adoption.

After the test methodology and tools are set, the next step is to put the right people in place with the proper skills and training to do the work. The key to automation success is to focus your resources on the test production. That is to improve the quantity and quality of the tests, not to spend too many resources on automation production.


The most essential element to achieve these benefits is the methodology, not the tools.


In evaluating the return on investment with test automation, you need to look at the big picture. Think of the return-on-investment (ROI) equation, with the benefits on one side, and the costs on the other. For the benefits, consider the productivity, both in quantity and quality of tests. For the cost side of the equation, think about the reusability, scalability and maintainability of the tests in the context of the phases of a test automation effort:

  • Deployment costs
  • Test automation creation costs
  • Execution costs
  • Maintenance costs

Evaluate the ROI by considering if the costs are justified by:

  • Faster/more tests?
  • Faster/more test cycles?
  • Better test coverage in each cycle?
  • Higher quality of tests?

To make these evaluations, you need to have good visibility.

Often, the costs and benefits of test automation are uncertain due to lack of visibility. Management just does not know how much money it will spend on it, and how much benefit it will get from it. People are uncertain about how to quantify the ROI and how to set up a way to monitor it to see if they are on target. This is symptomatic to starting a project, and six months later something gets derailed.


Excellent visibility leads to effective management of test automation production.


Defining the test methodology before choosing the right enabling technology should provide measurable improvements in visibility, reusability, scalability, and maintainability yielding many tangible benefits from test automation efforts. To read more about this, see LogiGear’s book Global Software Test Automation: An Executive Snapshot of the Industry.

LogiGear Corporation

LogiGear Corporation LogiGear Corporation provides global solutions for software testing, and offers public and corporate software-testing training programs worldwide through LogiGear University. LogiGear is a leader in the integration of test automation, offshore resources and US project management for fast and cost-effective results. Since 1994, LogiGear has worked with hundreds of companies from the Fortune 500 to early-stage startups, creating unique solutions to exactly meet their needs. With facilities in the US and Vietnam, LogiGear helps companies double their test coverage and improve software quality while reducing testing time and cutting costs. For more information, contact Joe Hughes + 01 650.572.1400

LogiGear Corporation
LogiGear Corporation provides global solutions for software testing, and offers public and corporate software testing training programs worldwide through LogiGear University. LogiGear is a leader in the integration of test automation, offshore resources and US project management for fast, cost-effective results. Since 1994, LogiGear has worked with Fortune 500 companies to early-stage start-ups in, creating unique solutions to meet their clients’ needs. With facilities in the US and Viet Nam, LogiGear helps companies double their test coverage and improve software quality while reducing testing time and cutting costs.

The Related Post

Has this ever happened to you: You’ve been testing for a while, perhaps building off of a branch, only to find out that, after all of this time, there is something big wrong. It’s a bad build and now you have to go backwards, fix something, and get a new build. Basically, you just wasted ...
How to do UI test automation with the fewest headaches I’m currently interviewing lots of teams that have implemented acceptance testing for my new book. A majority of those interviewed so far have at some point shot themselves in the foot with UI test automation. After speaking to several people who are about to do ...
Looking for a solution to test your voice apps across devices and platforms? Whether you’re new or experienced in testing voice apps such as Alexa skill or Google Home actions, this article will give you a holistic view of the challenges of executing software testing for voice-based apps. It also explores some of the basic ...
Take 5 and test your knowledge on the Software Testing essentials covered in our March 2019 issue. Good luck!
“Happy About Global Software Test Automation: A Discussion of Software Testing for Executives” Author: Hung Q. Nguyen, Michael Hackett, and Brent K. Whitlock Publisher: Happy About (August 1, 2006) Finally, a testing book for executives!, November 17, 2006 By Scott Barber “Chief Technologist, PerfTestPlus” Happy About Global Software Test Automation: A Discussion of Software Testing ...
Mobile usage today is not just a trend but it is an essential shift in how people communicate with each other, interact with the world, and do business. According to a ComScore, in 2014 the number of mobile users surpassed the number of computer users and is showing strong growth over time, towards some point in ...
Elfriede Dustin of Innovative Defense Technology, is the author of various books including Automated Software Testing, Quality Web Systems, and her latest book Effective Software Testing. Dustin discusses her views on test design, scaling automation and the current state of test automation tools. LogiGear: With Test Design being an important ingredient to successful test automation, ...
A short-list of selection criteria and popular automation tools. There are a lot of test automation tools available in the market, from heavy-duty enterprise level tools to quick and dirty playback-and-record tools for browser testing. For anyone just starting their research we’ve put together a short list of requirements and tools to consider.
People who know me and my work probably know my emphasis on good test design for successful test automation. I have written about this in “Key Success Factors for Keyword Driven Testing“. In the Action Based Testing (ABT) method that I have pioneered over the years it is an essential element for success. However, agreeing ...
The growing complexity of the Human-Machine Interface (HMI) in cars offers traditional testers an opportunity to capitalize on their strengths. The human-machine interface (HMI) is nothing new. Any user interface including a graphical user interface (GUI) falls under the category of human-machine interface. HMI is more commonly being used to mean a view into the ...
For this interview, we talked to Greg Wester, Senior Member Technical Staff, Craig Jennings, Senior Director, Quality Engineering and Ritu Ganguly, QE Director at Salesforce. Salesforce.com is a cloud-based enterprise software company specializing in software as a service (SaaS). Best known for its Customer Relationship Management (CRM) product, it was ranked number 27 in Fortune’s 100 ...
LogiGear Magazine – The Big Testing Issue – April 2012

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe