|
Add to del.icio.us
Digg it!
Add to reddit
Add to MyWeb
Automation Selection Criteria – Picking the "Right" Candidates (Part 1 of 3)
By Bob Galen, Founder RGalen Consulting Group, LLC
Introduction
Companies approach test automation in different ways. For some, they have done a little research and made the business case to upper management regarding test automation with upper management accepting the proposal. In this case, upper management supports the effort and gets extremely excited about how much faster testing can really move…
Or
Upper management puts forth an edict to start automating testing. They want to improve testing cycle and turnaround times and do it with fewer resources. They clearly believe automation is the only way to achieve these goals. They have issued a nearly blank check to quickly bring automation into play for the next major release - 3 months away…
In either case, you are in an enviable position poised to begin developing test automation. Automation can provide a tremendous boost to the test team in technical challenge, contribution to the organization, while providing an opportunity to wow the business folks in driving testing cycle times down and coverage up. Frankly though, this can also be an intimidating time - especially if this is your first time trying to sort out where and how to begin, which is exactly the focus of this article series.
One of the basic assumptions for this series is that you have been creating test cases and manually testing as an organization for a while. That is, you have built up some sort of repository of manual test cases that are potential automation "candidates". Given that you can not automate everything at once, the question of where to start and how to properly orchestrate your efforts over time becomes a challenge.
It also assumes that you do not have infinite resources or time to produce visible results. That is, you have other testing responsibilities besides the automation, for example testing and releasing your products. So prioritization and establishing a work balance becomes a challenge as well.
The series will examine three key areas to help you craft a strategy to meet these challenges:
- First, a few common anti-patterns that impede good test case selection.
- Then, a solid set of good practice patterns for test case selection.
- And finally, a wrap up on prioritization and criteria adjustment factors so that you can truly be nimble over time.
This, the first article in this series, will explore the anti-patterns that impede good test case selection.
Common Anti-Patterns for "Starting" Automation
Anti-patterns have been applied to software design, coding, configuration management, and just about every activity within software projects. Exposing what not to do is useful in setting the stage for some of the recommendations that will be made later on, so here are a few general anti-patterns for selecting good test candidates for automation development.
We Don't Need No Stinkin' Criteria
It is amazing how many groups simply dive into automation development without a strategy surrounding what test cases to automate. They simply start automating somewhere within their pool of test cases, often picking an arbitrary starting point such as first character in the tests name, and then moving serially forward from that point. Another part of the No Criteria anti-pattern is never reevaluating your lack of criteria as you make forward progress.
A Good Start – But What is Next
I also see teams who start well, for example, picking a set of "low hanging fruit" automation candidates that make sense to initiate the automation program. Typically the set is small, intended to get the effort going quickly and to realize some short term successes. However, after the team accomplishes their initial plans, they fall back into a no criteria, select anything you want mentality towards automation.
By George, Let's Do It All
Another frequent anti-pattern is the view that all test cases need to be automated. This creates churn because the team is frenetically trying to do everything. Frequently working in parallel with a mainline software project and struggling to automate test cases on a moving functional target. It also drives the mentality that all test cases need to be automated independent of the level of effort or returned value associated with that endeavor. Which is simply not business savvy nor realistic.
In Tools We Trust
This anti-pattern is focused towards the myriad of automation tools available. Often they can lull the team into this false sense of security that the tool will take care of the details surrounding automation - thus removing the need to understand the tools and technologies being used. It also masks the teams from understanding the strengths and weaknesses of each tool as it relates to different technologies. Every tool has "problems" that need to be considered.
Make A List, And Then Stick To It!
In this final anti-pattern, teams do everything well at the beginning. They take an inventory of their test cases and pull together a thoughtful ordering that starts building good automation and contributes positively to their projects. However, they become stuck in their initial list and never adjust it for discoveries and changes as they progress. As time moves on, their original approach becomes more and more irrelevant.
Conclusion
Clearly these anti-patterns set the stage for the remainder of the articles in this series - changing the focus towards what to do regarding selecting the best candidates for automation and developing your overall implementation strategy.
The next article in this series will examine a solid set of good practice patterns for test case selection.
About the Author
Bob Galen is the author of Software Endgames (Dorset House, 2005). He is also the founder of The RGalen Consulting Group, a technical consulting company that focuses on agility and pragmatism. Bob has over 25 years of experience as a software developer, tester, project manager and leader.
Other Articles by this Author
Download free articles, white papers, templates and more!
|
