Add to del.icio.us Digg it! Add to reddit Add to MyWeb

Developing a Business Case for Test Automation
(Part 3 of 3)

By Bob Galen, Founder RGalen Consulting Group, LLC

Introduction

The purpose of this series of articles is to share the breadth of factors that make for a compelling business case for test automation. The breadth of factors that make up a good business case for test automation include:

1. Assessing the climate for test automation
2. Exploring and defining Hard ROI goals for the effort
3. Exploring and defining Soft ROI goals for the effort

The first article in this series covered the most common assessment areas to examine when reviewing the automation business climate. The second article in this series covered determining the "hard" return on investment (ROI) in some detail. In this, the third and final article in this series, the "soft" ROI will be covered in detail.

Determining "Soft" ROI

There are two principles that have come out of the Lean Software Development thinking spearheaded by Mary & Tom Poppendieck. The first is to deliver features as late as possible, and the second is to deliver them as quickly as possible. This leads us to a "Just in Time and Just Enough" view of development that implies a focus toward real and demonstrable value rather than expected or anticipated value.

However, to execute on this vision, development teams need the capacity to make software changes quite late and still retain their overall product quality goals. Test automation is one of the few ways to provide this late change quality safety net and still allow the development team to remain as nimble as possible.

Both interplay nicely with a focus towards soft ROI. The three primary aspects supporting soft ROI are raw incremental speed, developer confidence, and reinvestment.

Raw Incremental Speed

Incremental speed can be a powerful differentiator in itself. Agile practices can be used to illustrate the point. There is a practice in most agile teams called Continuous Integration (CI). In CI, the team builds and runs a set of unit tests upon every check-in of code. The point is to continuously integrate every small change and catch any integration issues immediately - then repair them.

This practice turns traditional integration planning and work on its head. Instead of a big bang approach, the organization takes small, incremental steps that maximize discovery and minimize corrective rework. Test automation can enable the same incremental savings, but on the broader application feature set level, thus drastically increasing the overall quality of products, while reducing traditional testing integration cycle times.

Another factor that extends from speed is the notion of increasing testing coverage. In today's iterative development models, it is virtually impossible for test teams to test everything. However, increasing automation capabilities allows testing teams to iterate more quickly, testing more, and covering the remainder of applications with alternative testing techniques. For example, using Pareto based or other Risk Based techniques to more thoroughly test the high risk areas of the application to ensure overall product quality.

Development Confidence

One view of testing is that of providing a "safety net" for incremental development team application changes. Traditional software methods speak in terms of code freezes and change being the enemy - to largely influence diminishing changes. One of the drivers behind these views is that there was not a good traditional way to mitigate the side effects that late coming changes could incur.

However, a properly defined and well-implemented automation program can drastically influence development team confidence in making changes. The team can move toward incorporating late coming features that impact competitive advantage or refactoring the architecture when and where necessary. This confidence can lead to much more aggressive development action that embraces late-coming customer changes - resulting in more competitive product feature sets.

It can also lead to improved collaboration and gained mutual respect between the development and testing organizations - moving them towards more of a partnership role, than the adversarial one so often seen.

Continuous Investment

A final soft ROI factor is connecting the organization's savings to the continuous improvement of the testing team, which can take a couple of directions.

1. Time saved begets time - time that may be used to trim staff, or much more compellingly, used to train and improve the team and methods.
2. The time may also be used for tools investment and process improvement. For example, automation time saved may be reinvested in testing teams for training in Exploratory Testing (ET) techniques. ET can be a very effective, fast testing technique for application areas that are not yet automated or are not good automation candidates, even though you still want to test them non-traditionally and quickly.

Automation then becomes a fundamental part of the overall strategy for testing effectiveness AND team improvement. This logic can be extended towards team skill improvement, tools introductions, and larger scale process improvements.

The essence of the Soft ROI is to remind stakeholders that there are intangible but powerful benefits related to reinvesting potential savings within the testing team proper. That it can create an ongoing competitive advantage for technical projects - allowing for late and just enough application changes that can mean the difference between mediocrity and competitive advantage. Soft ROI advantages can impact organizations in much broader and profound ways than their Hard ROI counterparts.

Conclusion

Probably the most important part of a good Business Case is the assessment process (see Part 1 of this article set). It not only serves as a mechanism to gather critical focus points and key goals, but it also serves to establish the major risks that the organization will encounter along the way.

It is not uncommon for stakeholders to be frustrated with their automation efforts. They are frustrated with the unexpected costs and length of time it takes to make a visible impact. This frustration can reach a level where they want to take drastic actions within their automation efforts, often looking to throw away existing efforts and start again, or perform the work elsewhere - usually offshore.

To some degree stakeholders are looking for a silver bullet when it comes to automation. The root cause of all of this frustration connects back to the development of a clear Business Case that establishes the requirements, goals, timing, effort, costs, and ultimately the possible return for automation. Usually stakeholders did not focus on establishing a proper case or understanding what was truly feasible.

It is important to remember that once an organization understands the landscape, then they will need to effectively balance across Hard and Soft criteria to measure effectiveness. The organization should never make bold early promises to cut staff by large percentages before they have some automation experience runway underneath them!

Stakeholders often make another mistake - assuming that ROI is simply a cost cutting exercise. While cost cutting and/or increased revenue will always be an important part of the discussion, the view should be reframed toward ROI being a quality improvement and competitive advantage exercise instead. It is a much more powerful and contagious model.

About the Author

Bob Galen is the author of Software Endgames (Dorset House, 2005). He is also the founder of The RGalen Consulting Group, a technical consulting company that focuses on agility and pragmatism. Bob has over 25 years of experience as a software developer, tester, project manager and leader.

Other Articles by this Author

• Developing a Business Case for Test Automation (Part 1 of 3)
• Developing a Business Case for Test Automation (Part 2 of 3)

Download free articles, white papers, templates and more!

LogiGear's RSS feed