|
Add to del.icio.us
Digg it!
Add to reddit
Add to MyWeb
Establishing an Automation Development Lifecycle (Part 2 of 2)
By Bob Galen, Founder RGalen Consulting Group, LLC
Introduction
All automation efforts have to integrate with a development project. This means instead of having a stand-alone project, an organization has two interrelated development projects running in parallel, with automation understandably taking the priority back seat to its development counterpart. In the first article of this series, we discussed establishing the major drivers for creating an Automation SDLC, and exploring a few of the key success criteria behind a solid Automation SDLC effort, in an effort to help organizations to "connect" their automation efforts to traditional SDLC activities.
In this, the second and final article in this series, we go on to discuss: reviewing automation SDLC extensions from the organization's own product SDLC, and considering how to integrate automation correctly with mainline development efforts.
Review Automation SDLC Extensions from an Organization's Own Product SDLC
Surely an organization will not want to reinvent their own life-cycle for automation unless it absolutely has to. One reason for that is that automation is inextricably linked to their development methodology. For example, if they are using one of the Agile methodologies for product development, say Extreme Programming, their automation focus will be heavily influenced by its short iterations and acceptance testing demands.
Another reason is simply to avoid confusion. Having separately defined methods will often require ongoing explanation across project teams. It will also require an organization to define some sort of integration view between the two methods. It will also drive Project Managers crazy as they try to schedule across the two different perspectives.
Instead, an organization should extend their existing SDLC methods to include considerations for automation development. This is not too onerous because they typically already include testing activity anyway. This section discusses a few of the more critical parts of the extension model for automation - thus creating a group understanding of the Automation SDLC.
Extension #1, Make Automation Decisions from Completed Manual Test Cases
An organization should not change their manual test case development process for automation. In fact, they might even want to tighten it up a bit. Only when the manual cases have been developed, executed, and proven to be correct, should they analyze the need to automate and schedule them for automation development. There are several reasons for this. First, it helps an organization to have a consistent format and workflow for developing all of their test cases. It ensures that the test cases are essentially "complete" before worrying about automation. It also allows them to queue up a set of well defined work items that can be rearranged as priorities and needs change.
Consider manual test case development the early design phase for automated test cases. It will also sort out many requirements clarity issues before the organization ever starts coding. Finally, if there are parts of features that are not functional or are problematic, at least the organization will be aware of them in advance of automation development.
Extension #2, Review from an Automation Perspective
This assumes two review points in an organization's traditional process:
- That testers are invited to requirements review sessions
- That testers expose their test cases for external review as part of their test development process.
Even though an organization is deferring automation development decisions, it does want to surface automation concerns throughout the artifact review process. Often, automation requires data and environmental control that may drive product enhancements to truly enhance the effectiveness of the automation. An organization will want to get these on the table early on.
The organization is also looking for prioritization information on testing attributes for speed of execution and cycle time and or coverage, so that they can focus their automation efforts in higher impact areas. The overall product technical risk areas should also come into play - driving automation decision-making.
When reviewing requirements, an organization will want to ask questions surrounding feature dependencies and testability. As they begin to more heavily automate, they will notice more synergy between development unit testing and their automation efforts. The reviews are a wonderful place to share ideas, techniques and tools around testing each feature. The key to good automation requirements definition is to begin blurring the lines a bit between the development and testing perspectives.
Extension #3, Establish a Highly Iterative Automation Model
One thing that has been learned from operating in Agile teams, is that short iterations work extremely well in providing feedback on direction in software projects. If this is true in software development, it is even more applicable when developing test automation.
When developing any automation effort, an organization is stuck behind 2 requirement curves - the product and the automation. They are also trying to keep up with the implementation evolution for the product. These three forces create a lot of uncertainty and drive up risk. The ONLY way to combat those risks is with a finely grained iterative model.
One of the challenges facing that model is integrating automation with the product release stream. Rarely do feature delivery and functionality align perfectly to support automation efforts. An organization needs to become quite creative at negotiating early, finely grained functional releases from development whose primary purpose is to help their automation efforts. Call them automation Alpha candidates or something similar - to foster a willingness to share early and often.
This has a two-fold benefit. It allows the automation to be developed in small pieces iteratively, while also surfacing product defects early as well. It has a downside in that implementing automated tests on unstable features which may drive higher degrees of rework, but that is usually an acceptable tradeoff to automation deployment speed.
Extension #4, Establish Clear Hand-offs for Infrastructure, Automation, and Execution
From a workflow perspective, it is particularly important to have automation hand-offs defined properly. Hand-offs are the internal release of automation packages within an organization's teams. An organization wants software to be delivered for testing with release notes, exit/entry criteria, and in some sort of versioned package that has been tested. These same sorts of workflow process exchanges need to be planned and executed for automation deployment as well.
If an organization has opted in for a 3-phase automation model (automation infrastructure, automation development, and automation execution), then these hand-offs should align across those boundaries:
- Infrastructural components are created or modified, packaged and released to the automation development team (or individuals) for test development
- Completed, tested and packaged automated test cases are released to the automation execution team (or individuals) for execution
- The regression or automation execution team (or individuals) run the automation and provide feedback to the test developers for enhancements, efficiency / speed improvements, or general bug fixes
Following this workflow not only helps to manage expectations and planning for project support, but it also defines roles and expectations for deliverable quality within the automation development team.
Extension #5, Extend Traditional Tools & Processes for Automation Support
Many teams forget that automation requires many of the same tools and practices associated with production software development. A good example of this is the configuration management / version control, bug reporting, and change control tool sets and processes. Rarely is it a good idea to create separate environments for these tools.
Instead, an organization will want to extend their existing tools to include automation development as a project or series of projects. They will need naming and version conventions for their automation development. They will also want to get good at prioritizing defects (in automated tests) with customer input (from the test runners). Planning and packaging is important as well. On larger automation efforts it may make sense to instantiate more formal change control and a Change Control Board, just to bring some rigor and control to the automation development process.
The more reuse of tools, techniques and process an organization can create here, the more efficient their automation efforts will become simply because they do not have to train their team or establish unique tools and processes. They can even use examples to "lead" the product organization by exposing best practices from the automation team and suggesting they be adopted more broadly.
Extension #6, Look to Analyze, Increase & Communicate Automation Run-time Efficiency
As an organization's automation development matures and the overall coverage increases, analysis becomes more and more important from a project management delivery perspective. Thus the PM's will take more and more interest in plans, schedules and overall execution performance. They will want to take the time to explain to stakeholders the nuance of the organization's automation strategy. They will want to discuss where the strategy can increase overall testing efficiency and where it can not – and work to tailor plans to maximize project impact and success.
The organization will want to share some of the challenges associated with running automation, including automation maintenance, debugging and fault isolation, test environment management, and tools integration. They should draw on the similarities to traditional software development as a means of effectively communicating automation state.
The organization will also want to share efforts at making automation run-time more efficient or how it can actually become a speed differentiator in supporting development projects. This becomes an imperative if the organization is doing any outsourcing or implementing distributed automation development and/or execution.
While stakeholder communication is important at the beginning of automation efforts, it should be an ongoing priority and an extremely important part of the automation project manager's role. It is not at all uncommon for stakeholders to forget about the value proposition and nuance of test automation. The will need to be reminded often.
Integrating Automation Correctly with Mainline Development Efforts
It is important to note that any automation effort is deployed within the context of a higher priority project - the development project it is associated with. This brings into play an interesting set of dynamics where the two are inextricably linked, but also where one clearly outshines the other.
Typically what happens, and it should happen, is that the automation delivery is skewed or phased outside of the mainline product development activity. The testing team becomes responsible for testing the product (the prime directive) while trying to develop automation on early stabilizing and maturing features. The degree to which they can develop this automation AND have it deployed and usable within the current product iteration is dependent upon several factors:
- The organization's software product development life-cycle
- How quickly features within the software are maturing
- Whether the software development team is "on schedule"
- Overall resources available to the SQA team and
- Whether the testing team is "on schedule"
Essentially, an organization will deliver automation one release behind the product delivery schedule. This allows for features to stabilize and sufficient time for proper automation development and deployment into regression testing cycles. It also keeps automation development off the critical path of the primary development project.
While an organization can achieve compression of automation into the current release, it requires many of the factors listed above to be mitigated. It also is far more resource intensive if the organization tries to skew the automation work to be truly in parallel with product development.
Conclusion
The key focus of this article series has surrounded developing an organization's own view to an Automation–SDLC, but not by reinventing the wheel. Instead by leverage their existing product development SDLC and extending its tactics, processes, tools, and practices to include automation development.
If organizational practices do not include solid development practices, for example chartering, architecture development, requirements definition, and solid reviews, then it should simply lead the way by implementing them from within their test automation development efforts.
Also, an organization should never lose sight that automation development is a software development activity and all of those same rules and methods should clearly apply. It should ensure that stakeholders understand this. Finally, while not formally explored within this article, an organization will never want their automation efforts to become a critical path item within their primary product development schedule. Instead, it should skew them out so that they have the time and focus to implement them correctly and without de-railing the project.
Establishing a connected and clearly defined Automation-SDLC is one of the first steps needed to be taken down the automation path.
About the Author
Bob Galen is the author of Software Endgames (Dorset House, 2005). He is also the founder of The RGalen Consulting Group, a technical consulting company that focuses on agility and pragmatism. Bob has over 25 years of experience as a software developer, tester, project manager and leader.
Other Articles by this Author
Download free articles, white papers, templates and more!
|
