Agile Testing: Key Points for Unlearning

When quality assurance teams and management who have adopted Agile practices first put the ideas to work, they face a significant impediment in unlearning the traditional mind-set and practices that experience in traditional practices has instilled in them.

“He who knows to unlearn, learns best.” — Anonymous

The following are some of the key aspects that need to be unlearned before attempting to deploy Agile practices from a QA perspective:

  • The testing team needs to be independent and independently empowered in order to be effective.
  • Without a separate test strategy and test plan, it’s tough to manage testing.
  • The V-model for verification and validation cannot be applied in an Agile sprint.
  • Independent testing teams don’t do white-box testing.
  • The value of testing is realized only when defects are logged.
  • Automation is optional and is required only when regression testing is needed.
  • Testing nonfunctional aspects, such as performance of the system, is not possible in a sprint.
  • Testing must follow planning, specification, execution, and completion sequentially.
  • We don’t have to write new test cases for detected defects.
  • Poorly written code is not the testing team’s focus, as long as the code addresses the required functionality.
  • Test-process improvement models do not address aspects of Agile testing.

Let’s look at these assertions one by one.

The testing team needs to be independent and independently empowered in order to be effective.

Traditionally, testing teams have had followed different organizational styles, from having no independent testers while developers perform the testing, to having independent testers within the development teams, to having independent testing performed by a separate division within the organization — even outsourcing independent testing. Often the testing team would like to be empowered and report directly to a senior project manager rather than to the development or the technical lead. The logic, or at least the perceived logic, is to allow the testing team to report and escalate technical defects without potential inhibitions from the technical lead. The Agile testing mind-set change that’s required is that testers are an integral part of an Agile team. Their focus is to deliver a quality shippable product at the end of each sprint and to achieve the “done” state for the backlog items committed without any technical debt. The testers report to the Agile team and are accountable to the product owner or the business.

Without a separate test strategy and test plan, it’s tough to manage testing.

A test strategy document can typically be defined at the organizational level, the division or portfolio level, or even at the product level. Seldom must the test strategy be defined for each project, unless the project is large and the duration spans many years. The project-specific test approach is documented in the test plan for the project.

In the case of Agile projects, the test approach can be documented in the release plan, and the sprint-specific testing activities during sprint planning. A separate test plan may not be required. However, having a test strategy at a level higher than the project could be useful, especially when the organization is undergoing transformation to Agile. The test strategy can define the Agile testing practices and the techniques to be followed across the organization or division; subsequently, Agile teams can adopt one or more of these practices while defining the test approach in the release plan for the particular project.

The V-model for verification and validation cannot be applied in an Agile sprint.

Within an Agile sprint, verification and validation are addressed by adopting Agile practices. This includes verifying whether INVEST criteria for documenting requirements is followed, creating and reviewing evocative documentation and simple design, reviewing visual modeling, holding daily stand-up meetings, reviewing radiator boards, following continuous integration, refactoring, running automated development tests and automated acceptance tests, holding focused reviews, and enhancing communication by having the product owner and customer on the team.

The following figure shows an Agile V-model for verification and validation, as compared to the traditional V-model (fig. 1).

Independent testing teams don’t do white-box testing.

Independent testing teams traditionally focus on black-box testing, possibly shrugging off any responsibility related to low-level testing. However, in Agile projects, testers play a significant role in automated development and acceptance tests. Agile testing is continuous and seldom staged. Agile testers need to understand the design and code-level aspects in order to effectively perform testing for a sprint. While the developers take the lead in unit testing, the Agile testing team shadows the low-level testing efforts and leads the automation aspect.

The value of testing is realized only when defects are logged.

While the value of testing lies in early detection of defects and ensuring that the shippable product is of good quality, Agile teams need to unlearn the defect numbers-game mind-set. Teams may perceive that more detected defects indicates better performance of the testing team. As a result, many cosmetic defects are logged.

The Agile testing team directly contributes to the “done” state of the product backlog item, which essentially means that a backlog item cannot be considered done unless it passes testing. Agile testing teams must make use of the radiator boards to effectively radiate the information on the status of the backlog items.

Automation is optional and is required only when regression testing is needed.

Automation is not optional; it’s an essential aspect, especially when the business is trying to improve the time to market for its products. Agile teams working at peak velocity adopt such practices as continuous integration, automated development tests, and automated acceptance tests. Without automation, the team cannot achieve the desired agility.

Testing nonfunctional aspects, such as performance of the system, is not possible in a sprint.

Sometimes it may not be possible to perform testing of nonfunctional aspects, such as system performance, within a sprint. However, this can be addressed by having a separate release sprint during release planning. The release sprint can address the required nonfunctional testing and also perform a cycle of acceptance testing to ensure that the system works after any defect fixes. Rigorous integration testing may not be required if the system was continuously integrated and tested by leveraging automation.

Testing must follow planning, specification, execution, and completion sequentially.

The aspects of planning, specification, execution, and completion are highly relevant in Agile testing. However, we need to understand that Agile testing is continuous, not staged. While one backlog item may be marked “done,” another item could be in its specification stages. Some teams follow the practice of updating a backlog item as done only when the test cases are automated for the backlog item.

We don’t have to write new test cases for detected defects.

Traditionally, it hasn’t been a practice for a test team to go back to specify a test case for a detected defect, especially for defects detected during exploratory testing. One of the key pain points for not doing so is the process of re-baselining the test case document and running around for signatures, since this is a change from the planned baseline. However, adapting to change is one of the Agile framework’s foundational aspects. In Agile testing, new test cases are specified for detected defects that don’t already have an associated test case, and the test case is subsequently included in the automation test cases suite.

Poorly written code is not the testing team’s focus, as long as the code addresses the required functionality.

This is related to the point above (“Independent testing teams don’t do white-box testing”). Independent test teams traditionally focus only on black-box testing and may be unconcerned with the quality of the code as long the code performs the required functionality. But the value add from the testing team can be significant if it can provide early feedback and also identify technical debt by focusing on the code-level aspects during verification and during validation or testing. This is one of the key Agile-testing mind-set changes required for a new Agile tester.

Test-process improvement models do not address aspects of Agile testing.

In fact, we do have the ability to measure and improve Agile testing – using standard industry models. Test-process
improvement methods such as TPI NEXT advocate business-driven test process improvement in an Agile environment by prioritizing the key areas of focus. This facilitates an Agile testing mind-set by mapping Agile principles with specific, prioritized areas. TPI NEXT also provides specific “improvement suggestions” for the checkpoints in priority areas of Agile testing.

Conclusion

Although the task of performing testing is not very different in principle in waterfall, iterative, or Agile, the Agile mind-set and its testing practices provide effective new means to achieve the desired results. The agility lies in the Agile
practices, rather than in the overarching process itself.

This article was originally published on the Scrum Alliance website (http://www.scrumalliance.org).

Madhu Expedith
Madhu Expedith has 12 years of combined experience in IT process and quality consulting, project management, quality management and software development. Madhu is currently an IT manager and principal consultant in the process and quality consulting practice, Enterprise Quality Solutions (EQS). He has executed projects involving end-to-end process definition, implementation, quality management, driving and managing organizational process changes, anchored process improvement programs including training & facilitation. His expertise includes models and frameworks such as Agile, CMMI, ITIL, TPI, TPINEXT, RUP and regulations such as GCP and 21 CFR Part11.

The Related Post

LogiGear Magazine July 2012 Testing in Agile  
Video narrated by MICHAEL HACKETT – Certified ScrumMaster This is Part One of a Four Part Video on “New Roles for Traditional Testers in Agile Development” Michael shares his thoughts on “A Primer – New Roles for Traditional Testers in Agile” LogiGear Corporation LogiGear Corporation LogiGear Corporation provides global solutions for software testing, and offers ...
How to fit automated testing into scrum, and keep testers in sync with other teams One of the benefits of the approaches of agile projects is their friendliness towards testing. The testing activities, and the testers with it, are integrated into the teams, and testing and quality are redefined as team responsibilities. Automation nowadays is a must-have ...
This article presents ten tips for Agile testing based on our experience. However, don’t expect to find the perfect test approach for your company or software project in this article. That is still something you will have to find out yourself! Several years ago I started as test manager on a J2EE project. The project ...
Agile, in terms of software development, has incorrectly and for too long come to mean fast and “getting product out the door quicker.” But Agile is not about speed; it is about being flexible. I always begin my discussions on Agile development by getting a definition for the word Agile. Agile, in terms of software development, ...
Mark Levison has over twenty years experience in the IT industry, working as a developer, manager, technical lead, architect, and consultant. He discovered Agile in 2001 and is now a Certified Scrum Trainer and Agile Coach with Agile Pain Relief Consulting. Levison has introduced Scrum, Lean and other Agile methods to a number of organizations and coaches from ...
Agile is a philosophy focused on delivering constant value to customers incrementally and frequently, based on communication and feedback. These two ingredients are vital to a successful Agile recipe. Agile is no longer a buzzword or an unknown territory in the industry. Agile has progressed leaps and bounds the last few years and has matured to ...
Build the right test platform including infrastructure, virtual lab and process. Testing embedded software is both similar and dissimilar to application software testing. The first eye-catching thing is that embedded software is significantly less visible to the end user. User interfaces are limited; there may be a console-based text menu, a simple command line interface, ...
There is a multitude of Agile testing techniques that are quite sophisticated. The DAD process can help guide your process of tailoring decisions. Agile developers are said to be quality infected, and disciplined agilists strive to validate their work to the best of their ability. As a result they are finding ways to bring testing and ...
Video narrated by MICHAEL HACKETT – Certified ScrumMaster This is Part Four of a Four Part Video on “New Roles for Traditional Testers in Agile Development” Michael shares his thoughts on “A Primer – New Roles for Traditional Testers in Agile”   LogiGear Corporation  LogiGear Corporation LogiGear Corporation provides global solutions for software testing, and ...
In the decade since the Agile Manifesto, the movement has encouraged a number of best practices like test-driven development, user-centered design, iterative development, clean code, refactoring, continuous integration, and—arguably—cloud computing. I’m a card-carrying Agile zealot, and to me its benefits are unarguable. Is your IT organization ready to be Agile, seriously? Score yourself on these ...
Janet Gregory draws from her own experience in helping agile teams address alternative ways to cope with roadblocks including projects without clear documentation, testers with limited domain knowledge and dealing with either black box or white box testing. For testing on projects without clear documentation, is exploratory the only method? I often make “tester errors” ...

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe