Agile Testing: Key Points for Unlearning

When quality assurance teams and management who have adopted Agile practices first put the ideas to work, they face a significant impediment in unlearning the traditional mind-set and practices that experience in traditional practices has instilled in them.

“He who knows to unlearn, learns best.” — Anonymous

The following are some of the key aspects that need to be unlearned before attempting to deploy Agile practices from a QA perspective:

  • The testing team needs to be independent and independently empowered in order to be effective.
  • Without a separate test strategy and test plan, it’s tough to manage testing.
  • The V-model for verification and validation cannot be applied in an Agile sprint.
  • Independent testing teams don’t do white-box testing.
  • The value of testing is realized only when defects are logged.
  • Automation is optional and is required only when regression testing is needed.
  • Testing nonfunctional aspects, such as performance of the system, is not possible in a sprint.
  • Testing must follow planning, specification, execution, and completion sequentially.
  • We don’t have to write new test cases for detected defects.
  • Poorly written code is not the testing team’s focus, as long as the code addresses the required functionality.
  • Test-process improvement models do not address aspects of Agile testing.

Let’s look at these assertions one by one.

The testing team needs to be independent and independently empowered in order to be effective.

Traditionally, testing teams have had followed different organizational styles, from having no independent testers while developers perform the testing, to having independent testers within the development teams, to having independent testing performed by a separate division within the organization — even outsourcing independent testing. Often the testing team would like to be empowered and report directly to a senior project manager rather than to the development or the technical lead. The logic, or at least the perceived logic, is to allow the testing team to report and escalate technical defects without potential inhibitions from the technical lead. The Agile testing mind-set change that’s required is that testers are an integral part of an Agile team. Their focus is to deliver a quality shippable product at the end of each sprint and to achieve the “done” state for the backlog items committed without any technical debt. The testers report to the Agile team and are accountable to the product owner or the business.

Without a separate test strategy and test plan, it’s tough to manage testing.

A test strategy document can typically be defined at the organizational level, the division or portfolio level, or even at the product level. Seldom must the test strategy be defined for each project, unless the project is large and the duration spans many years. The project-specific test approach is documented in the test plan for the project.

In the case of Agile projects, the test approach can be documented in the release plan, and the sprint-specific testing activities during sprint planning. A separate test plan may not be required. However, having a test strategy at a level higher than the project could be useful, especially when the organization is undergoing transformation to Agile. The test strategy can define the Agile testing practices and the techniques to be followed across the organization or division; subsequently, Agile teams can adopt one or more of these practices while defining the test approach in the release plan for the particular project.

The V-model for verification and validation cannot be applied in an Agile sprint.

Within an Agile sprint, verification and validation are addressed by adopting Agile practices. This includes verifying whether INVEST criteria for documenting requirements is followed, creating and reviewing evocative documentation and simple design, reviewing visual modeling, holding daily stand-up meetings, reviewing radiator boards, following continuous integration, refactoring, running automated development tests and automated acceptance tests, holding focused reviews, and enhancing communication by having the product owner and customer on the team.

The following figure shows an Agile V-model for verification and validation, as compared to the traditional V-model (fig. 1).

Independent testing teams don’t do white-box testing.

Independent testing teams traditionally focus on black-box testing, possibly shrugging off any responsibility related to low-level testing. However, in Agile projects, testers play a significant role in automated development and acceptance tests. Agile testing is continuous and seldom staged. Agile testers need to understand the design and code-level aspects in order to effectively perform testing for a sprint. While the developers take the lead in unit testing, the Agile testing team shadows the low-level testing efforts and leads the automation aspect.

The value of testing is realized only when defects are logged.

While the value of testing lies in early detection of defects and ensuring that the shippable product is of good quality, Agile teams need to unlearn the defect numbers-game mind-set. Teams may perceive that more detected defects indicates better performance of the testing team. As a result, many cosmetic defects are logged.

The Agile testing team directly contributes to the “done” state of the product backlog item, which essentially means that a backlog item cannot be considered done unless it passes testing. Agile testing teams must make use of the radiator boards to effectively radiate the information on the status of the backlog items.

Automation is optional and is required only when regression testing is needed.

Automation is not optional; it’s an essential aspect, especially when the business is trying to improve the time to market for its products. Agile teams working at peak velocity adopt such practices as continuous integration, automated development tests, and automated acceptance tests. Without automation, the team cannot achieve the desired agility.

Testing nonfunctional aspects, such as performance of the system, is not possible in a sprint.

Sometimes it may not be possible to perform testing of nonfunctional aspects, such as system performance, within a sprint. However, this can be addressed by having a separate release sprint during release planning. The release sprint can address the required nonfunctional testing and also perform a cycle of acceptance testing to ensure that the system works after any defect fixes. Rigorous integration testing may not be required if the system was continuously integrated and tested by leveraging automation.

Testing must follow planning, specification, execution, and completion sequentially.

The aspects of planning, specification, execution, and completion are highly relevant in Agile testing. However, we need to understand that Agile testing is continuous, not staged. While one backlog item may be marked “done,” another item could be in its specification stages. Some teams follow the practice of updating a backlog item as done only when the test cases are automated for the backlog item.

We don’t have to write new test cases for detected defects.

Traditionally, it hasn’t been a practice for a test team to go back to specify a test case for a detected defect, especially for defects detected during exploratory testing. One of the key pain points for not doing so is the process of re-baselining the test case document and running around for signatures, since this is a change from the planned baseline. However, adapting to change is one of the Agile framework’s foundational aspects. In Agile testing, new test cases are specified for detected defects that don’t already have an associated test case, and the test case is subsequently included in the automation test cases suite.

Poorly written code is not the testing team’s focus, as long as the code addresses the required functionality.

This is related to the point above (“Independent testing teams don’t do white-box testing”). Independent test teams traditionally focus only on black-box testing and may be unconcerned with the quality of the code as long the code performs the required functionality. But the value add from the testing team can be significant if it can provide early feedback and also identify technical debt by focusing on the code-level aspects during verification and during validation or testing. This is one of the key Agile-testing mind-set changes required for a new Agile tester.

Test-process improvement models do not address aspects of Agile testing.

In fact, we do have the ability to measure and improve Agile testing – using standard industry models. Test-process
improvement methods such as TPI NEXT advocate business-driven test process improvement in an Agile environment by prioritizing the key areas of focus. This facilitates an Agile testing mind-set by mapping Agile principles with specific, prioritized areas. TPI NEXT also provides specific “improvement suggestions” for the checkpoints in priority areas of Agile testing.

Conclusion

Although the task of performing testing is not very different in principle in waterfall, iterative, or Agile, the Agile mind-set and its testing practices provide effective new means to achieve the desired results. The agility lies in the Agile
practices, rather than in the overarching process itself.

This article was originally published on the Scrum Alliance website (http://www.scrumalliance.org).

Madhu Expedith
Madhu Expedith has 12 years of combined experience in IT process and quality consulting, project management, quality management and software development. Madhu is currently an IT manager and principal consultant in the process and quality consulting practice, Enterprise Quality Solutions (EQS). He has executed projects involving end-to-end process definition, implementation, quality management, driving and managing organizational process changes, anchored process improvement programs including training & facilitation. His expertise includes models and frameworks such as Agile, CMMI, ITIL, TPI, TPINEXT, RUP and regulations such as GCP and 21 CFR Part11.

The Related Post

In the decade since the Agile Manifesto, the movement has encouraged a number of best practices like test-driven development, user-centered design, iterative development, clean code, refactoring, continuous integration, and—arguably—cloud computing. I’m a card-carrying Agile zealot, and to me its benefits are unarguable. Is your IT organization ready to be Agile, seriously? Score yourself on these ...
Testing is often looked upon by many as an unmanageable, unpredictable, unorganized practice with little structure. It is common to hear questions or complaints from development including: What are test teams doing? Testing takes too long Testers have negative attitudes
SKILLS Agile teams need training! One of the missing links in the implementation of Agile development methods is the lack of training for teams. I noticed in our recent survey on Agile that only 47% of the respondents answered “Yes” that they had been trained in the Agile development process, with over half responding “No.” ...
This is part 1 of a 2-part series. The 1st part will discuss the culture and mindset around Agile, and how Agile Quadrants are used. Part 2 will discuss how to use the Agile Quadrant, the significance of Automation in Agile Quadrants and how to use Agile Quadrants to overcome Quality Assurance headaches. Organizations aspire ...
If your Agile implementation is not about people, you’ve missed the boat! The most profound impact to becoming more Agile is happier teams! Agile manifesto Value #1: * Individuals and interactions over processes and tools Words like these do not show up in Waterfall or RUP SDLC process descriptions. Agile cannot get more basic than ...
One of the features of using Agile methods is the opportunity for continuous improvement within a project. There are a number of improvement opportunities throughout a typical iteration or sprint─over the next few weeks I’m going to walk through a few, starting this week with the Retrospective. Retrospectives are one of the many tools in ...
Author Jonathan Rasmusson explains in his latest book how to successfully set-up, execute and deliver Agile projects. Download the excerpt below for “Chapter 7: Estimation The Fine Art of Guessing.” To read his interview in last month’s issue, please click on “Spotlight Interview: Jonathan Rasmusson” to read his views on the best practices for test ...
Maximize the function of your teams The Modern Agile philosophy created by the folks at Industrial Logic is one of the most exciting ideas I’ve encountered in a while. Moving beyond the pre-canned “You must do X to be Agile” mindset that I’ve seen becoming more and more prevalent.
Build the right test platform including infrastructure, virtual lab and process. Testing embedded software is both similar and dissimilar to application software testing. The first eye-catching thing is that embedded software is significantly less visible to the end user. User interfaces are limited; there may be a console-based text menu, a simple command line interface, ...
LogiGear Magazine July 2012 Testing in Agile  
The sprint is almost over; the burn-down chart has not budged. The test team sits around waiting. They hear about all kinds of issues, obstacles and impediments at the daily stand-up but there is no code to test. Closing in on the demo and sprint review… then at Wednesday’s stand up: the heroes arrive and ...
Janet Gregory draws from her own experience in helping agile teams address alternative ways to cope with roadblocks including projects without clear documentation, testers with limited domain knowledge and dealing with either black box or white box testing. For testing on projects without clear documentation, is exploratory the only method? I often make “tester errors” ...

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe