Ten Tips for Agile Testing

This article presents ten tips for Agile testing based on our experience. However, don’t expect to find the perfect test approach for your company or software project in this article. That is still something you will have to find out yourself!

Several years ago I started as test manager on a J2EE project. The project team had switched from a waterfall approach to an Agile approach with Scrum a few months earlier. The first question the project manager asked me was, “Can you write a test plan for our first release?”

I quickly produced a project test plan that called for a test phase of a few months and a separate test team. It came complete with a capacity calculation per week for the testers, a MS-project document, and a matrix with all the quality attributes and the effort we should spent to test every attribute. What a mistake!

A few years later, we’ve learned a great deal about Agile testing. This article presents ten tips for Agile testing based on our experience. However, don’t expect to find the perfect test approach for your company or software project in this article. That is still something you will have to find out yourself!

1. Integrate the testers in the development teams

Teams are responsible for delivering software that meets expected requirements and quality. However, if we want teams to test the software, we must give them the knowledge to do it right. Testers have that knowledge. By integrating testers into the development teams, teams obtain the skills they need to test their software. When you try this, make sure you choose the right mix: one tester for three programmers is a fair but minimal number.

2. Use risk based testing

You can never test everything with the same (extensive) depth; even in a waterfall project you have to make choices. In an Agile project all the activities are time boxed so you have to make choices about how extensively you want to test each feature. We use a risk based approach to determine which test activities we are going to carry out for a feature during the iteration. The risk level of every feature is determined by the customer and the teams. It is a very transparent process so the customer knows exactly which test activities are executed for every feature.

3. Have testers review unit tests

In our organization the developers are responsible for creating and maintaining the unit tests. Unit tests are a very important part of our test harness. Developers often have a different way of thinking; for example, they tend to test only the success scenario.

To create unit tests that are as good as possible, our testers review the unit tests for all our high-risk items. The review has two advantages. First, the unit tests are improved because testers and developers supplement each other: the developer knows where the weak places in the source are, and the tester can use his knowledge of testing to give tips to improve the unit tests. Second, the testers know exactly which test cases are executed in the unit tests and can concentrate on executing other (e.g. higher-level) test cases.

4. Create a test automation framework and appoint a toolsmith

Automated testing is very important because new features and refactoring can introduce problems that can be difficult to find. By using an automated test framework, we can maintain quality levels during the iteration. Our testers are able to create new tests easily and quickly in the framework. We have a dedicated test engineer (we call him a tool smith) that maintains and optimizes the test automation framework, reviews the new automated tests of the testers and analyzes the daily test results. Testers in the teams can spend more time creating and extending automated tests because the toolsmith supports them.

5. Display quality metrics in a public location

Almost every software project has a problem registration system, automated test results, and in some cases nightly or continuous build results. But how often do team members look at the results or count the open problems? We installed a monitor in the coffee room that displays the actual metrics of the currently open problems, the percentage of successful unit tests, the percentage of successful nightly builds, and the current state of the continuous build. By displaying the metrics in public the teams are confronted with the information. The information is no longer just a number in a system or a record with some other information in a metrics database.

6. Add a test scrum

One advantage of having a separate test team in one room is that the communication between the testers is improved. When you have a project like ours where the testers are distributed across several teams, the communication becomes more difficult. To solve this problem, we use a test scrum to align the test activities. Our test scrum is held twice a week and every team is represented by one tester. The test scrum is a scrum like the daily team scrum but focused on test activities. The test manager is the ScrumMaster of the test scrum.

7. Implement test retrospectives

Every team in our project holds a retrospective meeting at the end of the iteration. In the retrospective, the team discusses the process: what went well and what went wrong. The testers in the team learn and discover new ways to improve their tests. The sharing of knowledge with testers from the other teams helps everyone.

We have a test retrospective every iteration so the testers can exchange knowledge and experience and discuss problems they have. It is important that the retrospective is only related to test issues; you shouldn’t discuss team issues (they should be discussed in the team retrospective). As with the test scrum, the test manager is the ScrumMaster of the test retrospective.

8. Plan for open problems

We try to fix all the problems that we find during the iteration in that same iteration, but sometimes we end the iteration with open problems. The best way to handle open problems is to add them to the sprint backlog for the next iteration. By explicitly planning the management of the open problems, the chance that they are “forgotten” and pile up is very small.

9. Remember: Testing is still testing

When you test in an Agile software project you can still use the “traditional” test techniques. We use exploratory testing but also apply test techniques such as boundary value analysis, equivalence partitioning, cause/effect diagram, and pair-wise testing. Which test technique we choose for a feature depends on its risk category. Exploratory testing is used in every category; but if the risk is higher we also apply more formal test techniques. The challenge for the tester is to use a formal test technique without delivering extensive test documentation.

10. Start doing it!

The last but most important tip is start doing it! Don’t talk too much about how you are going to introduce testers, or which test approach is perfect for your organization. You are going to make mistakes or discover that the chosen approach doesn’t fit your organization. That’s a given. But, the sooner you start, the sooner you can begin learning and improving your test approach. With the retrospective meetings, you have the opportunity to adapt your test approach every month.

Conclusion

We delivered the new release of our software that we began developing in January. During the last two weeks of May, we did some more exploratory testing, solved the remaining problems, and prepared the delivery. I wrote no extensive test plan, did no capacity calculation, nor create a matrix with quality attributes. We just started testing and improved our testing every month.

Ralph van Roosmalen
Ralph van Roosmalen has been working in the software industry since 1997. He started as software developer, has worked as project manager, test manager and development manager. He is currently Vice President Research and Development at RES Software. Ralph has written several articles about Agile testing and has spoken at several conferences on the topic of Agile testing and software development in general. RES Software products enable IT professionals to manage and deliver secure, personalized and compliant desktops independent of the underlying computing infrastructure – thin clients, virtual desktops, physical desktops, or server-based computing environments.
Ralph van Roosmalen
Ralph van Roosmalen has been working in the software industry since 1997. He started as software developer, has worked as project manager, test manager and development manager. He is currently Vice President Research and Development at RES Software. Ralph has written several articles about Agile testing and has spoken at several conferences on the topic of Agile testing and software development in general. RES Software products enable IT professionals to manage and deliver secure, personalized and compliant desktops independent of the underlying computing infrastructure – thin clients, virtual desktops, physical desktops, or server-based computing environments.

The Related Post

If your Agile implementation is not about people, you’ve missed the boat! The most profound impact to becoming more Agile is happier teams! Agile manifesto Value #1: * Individuals and interactions over processes and tools Words like these do not show up in Waterfall or RUP SDLC process descriptions. Agile cannot get more basic than ...
Video narrated by MICHAEL HACKETT – Certified ScrumMaster This is Part Four of a Four Part Video on “New Roles for Traditional Testers in Agile Development” Michael shares his thoughts on “A Primer – New Roles for Traditional Testers in Agile”   LogiGear Corporation  LogiGear Corporation LogiGear Corporation provides global solutions for software testing, and ...
As CTO of Xebia and highly experienced in offshore testing in India, Guido articulates his methods in addressing common challenges faced by the in-house and offshore teams. He weighs heavily on strategic tactics as well as key cultural aspects to execute efficient and effective Agile methods. 1. I work at a US-based company and we ...
Author Jonathan Rasmusson explains in his latest book how to successfully set-up, execute and deliver Agile projects. Download the excerpt below for “Chapter 7: Estimation The Fine Art of Guessing.” To read his interview in last month’s issue, please click on “Spotlight Interview: Jonathan Rasmusson” to read his views on the best practices for test ...
Agile is here to stay. Once the radical alternative to Waterfall development methods, these legacy methodologies are being disrupted and replaced by Agile practices that improve time-to-market, reduce development costs, and produce higher quality software that better meets customer expectations. As the world demands more software, development teams – from scrappy startups to big corporations ...
Testing is often looked upon by many as an unmanageable, unpredictable, unorganized practice with little structure. It is common to hear questions or complaints from development including: What are test teams doing? Testing takes too long Testers have negative attitudes
Michael Hackett sat down with FNC’s Chris Floyd to get his take on numerous Agile topics.
Video narrated by MICHAEL HACKETT – Certified ScrumMaster This is Part Two of a Four Part Video on “New Roles for Traditional Testers in Agile Development” Michael shares his thoughts on “A Primer – New Roles for Traditional Testers in Agile”  LogiGear Corporation LogiGear Corporation LogiGear Corporation provides global solutions for software testing, and offers ...
The sprint is almost over; the burn-down chart has not budged. The test team sits around waiting. They hear about all kinds of issues, obstacles and impediments at the daily stand-up but there is no code to test. Closing in on the demo and sprint review… then at Wednesday’s stand up: the heroes arrive and ...
Over the years many Agile proponents have come out strongly against offshoring some of the development team, and in particular against having a remote testing team. We made use of not one, but two separate outsourcing providers located in two distant locations. While we had many challenges, what we found was that by starting with ...
SKILLS Agile teams need training! One of the missing links in the implementation of Agile development methods is the lack of training for teams. I noticed in our recent survey on Agile that only 47% of the respondents answered “Yes” that they had been trained in the Agile development process, with over half responding “No.” ...
Our comprehensive issue on Agile, which was set to be released in June, has been moved to early July. We’ve made this decision in order to accommodate an article from one of our industry’s thought leaders. We’re really excited about this piece and we’re sure you will be too! LogiGear Magazine is dedicated to bringing ...

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe