April 2007 - Establishing an Automation Development Lifecycle (Part 1 of 2)
By Bob Galen, Founder RGalen Consulting Group, LLCIntroduction A frequent lament from individuals involved with an organization's automation efforts typically sounds something like this:
What these individuals and organizations have failed to understand is that automation is a software development effort. It takes time to start most worthwhile automation projects and many organizations struggle in the beginning at properly setting the stage. Another important aspect of the challenge is that all automation efforts have to integrate with a development project - so resources and schedules are often shared. So instead of having a stand-alone project, an organization has two interrelated development projects running in parallel, with automation understandably taking the priority back seat to its development counterpart. The intent of this article is to help organizations "connect" their automation efforts to traditional SDLC activities. While some test teams are getting better at it, far too many manage their automation outside of good software development practices. This trend needs to change much more aggressively. This article series will cover four key points:
This, the first of two articles in this series, will deal with the first two items: establishing the major drivers for creating an Automation SDLC, and exploring a few of the key success criteria behind a solid Automation SDLC effort. The follow-up or second installment in this article series will deal with the last two items: reviewing automation SDLC extensions from the organizations own product SDLC, and considering how to integrate automation correctly with mainline development efforts. Major Drivers for Creating an Automation SDLC There are three key drivers behind the need for an Automation SDLC:
Key Success Criteria Behind a Solid Automation SDLC Effort There are several considerations when defining and connecting automation efforts to an SDLC. While this section is not intended to be exhaustive, it will illustrate automation concerns that are quite similar to those encountered by software developers building great products. There are key success factors that an organization ignores at their own risk. Most of these are not a day-one problem and an organization can often skip many of them in the short term and gain a sense of progress and contribution. However, over the long term, a successful Automation SDLC must establish sufficient context surrounding these foundational points. Business Case The first thing to establish is the high level business case for automation. Part of this effort is establishing clear goals that link to the stakeholder and business expectations. See the article series Developing a Business Case for Test Automation for more on the components of a solid business case. Here we just focus on goals, from the cost, time and scope perspectives. That is, how much automation coverage (scope) is required, within what sort of timeframe and supporting what sort of project delivery targets (time) and limited by what budgetary (cost) factors. Another way of looking at the business case is developing a charter for the organization's automation project. The primary purpose of the charter or business case is to ensure that everyone is on the same page as to what the automation effort will actually do for the project or organization and, equally important, what it will not do. This is the first step to combating the confusion so often seen in key stakeholders who fundamentally do not understand the value proposition, challenges, and most importantly reoccurring costs associated with a solid automation program. Architecture Another important point of connection is architecture. While today's testing teams seem to be gaining in the ability to define proper automation framework architectures, it is still one of the larger challenges for most. Many still focus on creating automated scripts without ever thinking about defining an architecture that will support their development efforts longer term. Even if the organization purchases off-the-shelf tools, it will still need to wrap them with a thoughtful model to support automation efforts. Beyond architecture, an organization also needs to agree on the appropriate automated testing model for their product domain, skill set and requirements. Most of the leading approaches seem to fall into the following three categories:
As an organization goes down this list, the complexity and initial development effort goes up, while ongoing maintenance effort goes down. The more complex models also require much more development-centric test engineers to build the infrastructure. Requirements For testers, one of the greatest challenges surrounds the requirements gathering and management processes. All too often requirements are late in development, slow to solidify, and quick to change - often very late in the game. All the while code is quickly being written. It is one of the ongoing battles in most software projects. Therefore, most organizations have relevant experience with the need for requirements and the problems associated with the lack thereof. Organizations should not fall into this same trap when developing their test automation architectural framework and automated test cases. The need to take the time to develop a solid set of requirements, and ensure that they include key development and business stakeholders in this process so they collaboratively construct the automation capabilities vision. Implementation Strategy Rarely is it a good idea to start with test case #1 and automate through test case #10,000. Instead, an organization should come up with an automation implementation strategy that guides them towards their goals as established in their business case and requirements. Often the early strategy is simply learning and establishing their automation framework. Rarely is there much of a focus on return on investment at this stage. Instead an organization is simply experimenting to find the right automation tools and techniques that will work within their product domain and project culture. However, an organization will need to quickly redirect this strategy towards more traditional automation goals. For example increasing coverage, finding defects faster and improving overall production product quality. Project Synchronization This final success criteria extends from an organization's product development SDLC. Clearly automation efforts have to be effectively extended from the base product SDLC. While an organization can and should extend this into their Automation SDLC, which is the topic of the next section, they really need to understand the nuance, both technically and from a business and project management perspective, of each project they are attempting to automate. Conclusion All automation efforts have to integrate with a development project. This means instead of having a stand-alone project, an organization has two interrelated development projects running in parallel, with automation understandably taking the priority back seat to its development counterpart. In this article, the first of two on this topic, we discussed establishing the major drivers for creating an Automation SDLC, and exploring a few of the key success criteria behind a solid Automation SDLC effort, in an effort to help organizations to "connect" their automation efforts to traditional SDLC activities. The follow-up article in this series will go on to discuss: reviewing automation SDLC extensions from the organization's own product SDLC, and considering how to integrate automation correctly with mainline development efforts. About the Author Bob Galen is the author of Software Endgames (Dorset House, 2005). He is also the founder of The RGalen Consulting Group, a technical consulting company that focuses on agility and pragmatism. Bob has over 25 years of experience as a software developer, tester, project manager and leader. |
