The Importance of Volume in Automated Software Testing

All too often, software development organizations look at automating software testing as a means of executing existing test cases faster. Very frequently there is no strategic or methodological underpinning to such an effort. The approach is one of running test cases faster is better, which will help to deliver software faster.

Even in organizations that start to build a test automation effort on a good foundation of strategy, methodology, and supporting tools, taking such a narrow view of the test automation effort misses a huge dynamic.

Simply running existing test cases faster reduces testing time but does not increase testing capacity. Less time is taken to test, a good thing, but software testing may still be falling woefully behind software development and still represent a huge bottleneck. This is further exacerbated by the rapid increases in productivity that software development has undergone and continues to enjoy.

This approach can also lead to another detrimental dynamic. From this approach, often times organizations look to hand the test automation off to inexpensive low-skill organizations that have relatively little experience with testing and test automation – often times offshore. While this can work given the relatively modest goals for automation it creates a future problem. At some point in time the organization is going to come to realize that they need to substantially increase the volume of testing in order to keep pace with development, and they will have entirely the wrong team in place to accomplish this.

More than Automation, the Goal Should be High Volume Automation

To be truly successful in test automation requires a change in emphasis. While it is commendable to do the existing testing faster, the goal must be to do a very high volume of test automation. This will allow software testing to truly keep pace with development. By setting this as a goal, and putting in place the necessary processes, organization, and partners, a software testing organization can keep pace with development and make valuable contributions to software quality.

The Keys to High Volume Testing and Test Automation

What, then, are the keys to success in high-volume test automation? There are many key ingredients, including:

  • An overall software testing strategy that informs methodology and tool selection.
  • A methodology that supports the strategy and makes it easy to design and implement tests. As important, the methodology must also make it easy to maintain automated test cases. An organization does not want to trade the bottleneck of manual testing with a new bottleneck – maintaining test cases. Easy to maintain and reuse test cases are critically important to obtaining high levels of test automation and testing.
  • Software testing tools that support the strategy and methodology and make it easy to design, implement, maintain, and reuse test cases.

Another factor that can be critical to the success of such an effort is engaging the right partner to facilitate the process. Organizations that feel they are weak in the strategy and process department when it comes to software testing should look to a partner who can help them to align quality assurance objectives with business objectives. Such an organization can help to:

  • Define and refine objectives
  • Align objectives and business vision
  • Maximize test efficiencies with the latest methodologies and tools

Companies with a handle on strategy and process may wish to engage a partner who understands how to translate the vision into a high-volume testing program; one with experience implementing high-volume methodologies and the tools that support them. Equally as important is a partner with experience employing global resource strategies to help keep costs down.

Benefits of High Volume Testing

There are many benefits to a testing program that sets goals for very high volumes of developed, maintained, and executed test cases. These include:

  • A software testing organization that can keep up with the pace of development
  • Increased testing coverage
  • Improved software quality (assuming that good test cases are designed and automated)

Conclusion

Successful test automation programs do not simply aim to run existing test cases faster. Truly successful automated software testing programs set much more lofty goals. They set the goal of providing truly high volume testing and test automation. They implement the right strategies, methodologies, and tools to support high volume. Doing so, testing can keep pace with development and make big contributions to software quality.

Rob Pirozzi
Over 20 years of sales, marketing, management, and technology experience in high technology with exposure to industries including financial services, healthcare, higher education, government, and manufacturing; demonstrating a strong track record of success.

The Related Post

Check out the top 12 Automation tools with pros and cons–like Cross-Operating Systems, Cross-Automation Platforms, Programming Language Support, and more – for desktop Automation Testing. Although the demand for desktop app testing is not growing as fast as mobile and web app testing, it’s still a crucial day-to-day duty for many testers, especially those who ...
This book isn’t for everyone, but everyone can get some value out of it. What I mean by that rather confusing statement is that folks working in Agile environments will likely want to throw the book across the room while folks in more bureaucratic environments like CMMI or other waterfall environments will likely get a ...
Two dominant manual testing approaches to the software testing game are scripted and exploratory testing. In the test automation space, we have other approaches. I look at three main contexts for test automation: 1. Code context – e.g. unit testing. 2. System context – e.g. protocol or message level testing. 3. Social context – e.g. ...
LogiGear Magazine – January 2011 – The Test Automation Issue
Test execution and utility tools that can make your job easier My first exposure to the necessity for testers to have an array of tools was from the groundbreaking article “Scripts on my Toolbelt” by Danny Faught. Danny laid out the ideal approach to any testing job, and it got me thinking “How can I ...
LogiGear Magazine – April 2013 – Test Automation
The path to continuous delivery leads through automation Software testing and verification needs a careful and diligent process of impersonating an end user, trying various usages and input scenarios, comparing and asserting expected behaviours. Directly, the words “careful and diligent” invoke the idea of letting a computer program do the job. Automating certain programmable aspects ...
Has this ever happened to you: You’ve been testing for a while, perhaps building off of a branch, only to find out that, after all of this time, there is something big wrong. It’s a bad build and now you have to go backwards, fix something, and get a new build. Basically, you just wasted ...
Every once in a while a book is put together that should be read by every person with a relationship to software development. This book is one of them. Everyone dreams of automating their software testing, but few make it a reality. This down-to-earth book contains stories of 28 teams that went for it, including ...
LogiGear Magazine – April 2014 – Test Tool and Automation
A short-list of selection criteria and popular automation tools. There are a lot of test automation tools available in the market, from heavy-duty enterprise level tools to quick and dirty playback-and-record tools for browser testing. For anyone just starting their research we’ve put together a short list of requirements and tools to consider.
Identifying which tests to begin with when starting automation is key to driving testing cycle times down and coverage up. So there you are. You’ve done a little research and made the business case to upper management regarding test automation and they bit on the proposal. Surprisingly, they supported you all the way and are extremely ...

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe