Spotlight Interview with Jonathan Rasmusson

Author of The Agile Warrior, Rasmusson answers questions from LogiGear’s testing staff about test automation in agile projects.

As an experienced entrepreneur and former agile coach for ThoughtWorks, Rasmusson has consulted internationally, helping others find better ways to work and play together. When not coaching his sons’ hockey teams or cycling to work in the throes of a Canadian winter, Jonathan can be found sharing his experiences with agile delivery methods at his blog, The Agile Warrior. Developer by trade, Jonathan Rasmusson engages with LogiGear’s inquisitive staff to give us insight on how he addresses various agile test automation challenges. An excerpt from chapter seven of his book, titled “Planning Agile Projects’ Estimation – The Fine Art of Guessing” will be included in next month’s issue focused on Test Process Improvement. More excerpts can be found at The Pragmatic Bookshelf.

1. How does test case automation fit in among different sprints? How can you automate testing on new functionality during a sprint? How do you make decisions during a spring to automate a test or not?

Generally when creating software in agile we like to do our testing in the same iteration (or sprint) that the story is developed in–this ensures that what we’ve produced works. It’s more efficient than abandoning the story and picking it up again in the next iteration to complete the testing. So generally, it fits in towards the end of the iteration.

In having said that, getting all your automated testing done in a single iteration isn’t always possible. For example, at the start of a new project we may not have a mature automated testing strategy in place. It’s something we’ll probably discover and figure out as we go. But at a minimum, the unit tests should be there along with whatever else we can get going early in the project.

Deciding which tests to automate (and which not to) is a decision every team faces on their project. Unit tests are cheap and easy – every project should be writing a ton of these. Integration tests are a bit more complex and involve making trade-offs between costs, scalability and fragility. For example a nice end-to-end integration or smoke screen test can be extremely valuable for a team that just wants to know whether the latest push of their software into a development environment is working or not.

Automated GUI tests on the other hand can be extremely fragile and problematic. It’s not that you should never do them, you just have to be careful as you don’t want to waste a lot of teams on fixing a changing UI test because the GUI hasn’t settled down.

2. What type of test automation should we do in agile projects?

At a minimum, teams should be doing a lot of unit testing. These tests developers write to ensure that their software (at the method and class level) works as expected. Every team should be writing many of these.

After that, it really depends on the nature of your project and your team. If your system is primarily back-office, integration tests (tests verifying all your various subsystems) that are hooked up correctly can be extremely valuable. If you have a GUI / web front end, you may want some high-level smoke screen tests using a tool like Selenium or something.

3. What are all the challenges for a test automation team in agile?

There are a couple of challenges with test automation on any team. One is having the courage and discipline to do it. Automated testing is hard work. It means not moving forward and delivering new functionality. It’s hard for some people to stop, make the investment, and automate.

The second challenge of course is knowing what to automate. Teams need to figure and discover on each project which tests are going to pay off and which are going to give a lot of bang for your buck. We know we can’t automated everything (that would be too expensive) but we also know that not automating is going to kill us. So we just need to be flexible, try lots of different tests, keep what works, and drop anything that gets in the way.

4. About test automation script maintenance, should it be considered as a separate user story/ sprint?

No. While it’s OK for teams to reserve 10-20% of an iteration for general bug fixing, refactoring, or test script maintenance, I wouldn’t recommend turning it into a story. Why? Because it’s not something our customers would find very interesting. As much as we can, I prefer reserving stories for pieces of business functionality our customers would get excited about. Testing, refactoring, and general maintenance are generally things they don’t care about.

5. What is the best time to document testing in agile projects, at the beginning from just the user story or at the end after the function is complete and fully understood?

While I like the idea of automating tests at the start of the sprint, I have never been able to pull it off in practice. I think you could do it if you had a really good idea of what was required and a team with the people, skillset, and desire to do it. I just haven’t seen it yet. That would have to be a pretty mature and skilled team.

Also, quite often on new stories we don’t know what the end result is going to look like. It’s often not until the story is nearly complete, and we’ve gone through many iterations with the customer that we (including the customer) truly understand what is required─that makes automating ahead of time hard.

6. If other groups (marketing, development, project management) have cut back on documentation and measurement to become “leaner,” should testers still be required to write test cases, show traceability to user stories, write a test plan and produce a dashboard of metrics on test cases and bug counts?

I don’t see the connection. If one group in the organization is doing something different to become more lean or efficient– good for them. That’s to be applauded. Conversely if other groups aren’t being efficient or lean, that shouldn’t stop us from doing whatever it is we feel we need to do to serve our customers better.

So if writing test cases, tracking traceability, and extensive test plans aren’t adding any value, they should be dropped regardless of what others in the organization are doing.

7. If my team does not do a hardening sprint, when do we run a full regression test suite?

That’s something each team needs to decide for themselves. While I am generally not a fan of hardening sprints, they are sometimes required. The team just takes on a lot of technical debt, or that they start slowing down because they haven’t invested enough in their build and automation tools, so taking a sprint to firm these things up can help.

One obvious time to do a full on regression test is right before any major product release, e.g. a User Acceptance Testing (UAT). That’s something you’ll obviously want to do before going live. Having said that, I believe the goal of every agile project should be to make UAT redundant or a non-event. In other words, the team does such a good job with it’s automated and in house iteration testing, that when UAT comes around, no bugs are found; they’re not there. The team has been building and releasing software of such a high quality that it is no longer required.

I am not saying drop UAT. You’ve got to earn the right to do that. But every day more and more teams are making UAT a non essential activity instead of spending that time and money on more valuable things (improving and adding more features to the existing software).

8. I am on an agile team that does offshore testing. Often, I do not attend the daily stand-up due to time zones. Can agile be successful this way? What are some suggestions to help me be successful in my testing?

I am no expert in offshore testing. Not really my specialty. But I do find it interesting, that whenever projects get into trouble, they bring everyone together and put them in the same room. This is often the single greatest thing you can do to improve the productivity of your team. 

 

Jonathan Rasmusson

As an experienced entrepreneur and former agile coach for ThoughtWorks, Rasmusson has consulted internationally, helping others find better ways to work and play together. When not coaching his sons’ hockey teams or cycling to work in the throes of a Canadian winter, Jonathan can be found sharing his experiences with agile delivery methods at his blog, The Agile Warrior. Developer by trade, Jonathan Rasmusson engages with LogiGear’s inquisitive staff to give us insight on how he addresses various agile test automation challenges. An excerpt from chapter seven of his book, titled “Planning Agile Projects’ Estimation – The Fine Art of Guessing” will be included in next month’s issue focused on Test Process Improvement. More excerpts can be found at The Pragmatic Bookshelf.
Jonathan Rasmusson
As an experienced entrepreneur and former agile coach for ThoughtWorks, Rasmusson has consulted internationally, helping others find better ways to work and play together.

The Related Post

Distinguishing these terms from each other can be rather confusing. In an attempt to go back to the basics, Nadine Schaeffer explains in detail the benefits and the necessity of using realistic situations.
Writing code that is easy to read and easy to test is difficult to achieve. The fact that poorly written code can function often leads to coding practices that are effective but not necessarily efficient. Too often, many programmers fresh out of school write code in the manner that was effective for passing their courses, but contains ...
Agile is here to stay. Once the radical alternative to Waterfall development methods, these legacy methodologies are being disrupted and replaced by Agile practices that improve time-to-market, reduce development costs, and produce higher quality software that better meets customer expectations. As the world demands more software, development teams – from scrappy startups to big corporations ...
Maximize the function of your teams The Modern Agile philosophy created by the folks at Industrial Logic is one of the most exciting ideas I’ve encountered in a while. Moving beyond the pre-canned “You must do X to be Agile” mindset that I’ve seen becoming more and more prevalent.
If your Agile implementation is not about people, you’ve missed the boat! The most profound impact to becoming more Agile is happier teams! Agile manifesto Value #1: * Individuals and interactions over processes and tools Words like these do not show up in Waterfall or RUP SDLC process descriptions. Agile cannot get more basic than ...
Mark Levison has over twenty years experience in the IT industry, working as a developer, manager, technical lead, architect, and consultant. He discovered Agile in 2001 and is now a Certified Scrum Trainer and Agile Coach with Agile Pain Relief Consulting. Levison has introduced Scrum, Lean and other Agile methods to a number of organizations and coaches from ...
Armed with the right tool or set of tools, a development team can incorporate ALM into its Agile process and start reaping the benefits of Agile ALM. As the software development industry matures, it is devising methods for ushering products from inception to completion—a process that has come to be known by the buzzword ALM ...
Video narrated by MICHAEL HACKETT – Certified ScrumMaster This is Part Two of a Four Part Video on “New Roles for Traditional Testers in Agile Development” Michael shares his thoughts on “A Primer – New Roles for Traditional Testers in Agile”  LogiGear Corporation LogiGear Corporation LogiGear Corporation provides global solutions for software testing, and offers ...
Build the right test platform including infrastructure, virtual lab and process. Testing embedded software is both similar and dissimilar to application software testing. The first eye-catching thing is that embedded software is significantly less visible to the end user. User interfaces are limited; there may be a console-based text menu, a simple command line interface, ...
Video narrated by MICHAEL HACKETT – Certified ScrumMaster This is Part One of a Four Part Video on “New Roles for Traditional Testers in Agile Development” Michael shares his thoughts on “A Primer – New Roles for Traditional Testers in Agile” LogiGear Corporation LogiGear Corporation LogiGear Corporation provides global solutions for software testing, and offers ...
Video narrated by MICHAEL HACKETT – Certified ScrumMaster This is Part Four of a Four Part Video on “New Roles for Traditional Testers in Agile Development” Michael shares his thoughts on “A Primer – New Roles for Traditional Testers in Agile”   LogiGear Corporation  LogiGear Corporation LogiGear Corporation provides global solutions for software testing, and ...
Team collaboration is essential for testing embedded systems. Developing software for an embedded system often carries more risk than for general purpose computers, so testing is extremely critical. However, there still has to be a good balance between time spent on testing and time spent on development to keep the project on track. As consultants ...

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe