Book Review: Experiences of Test Automation

Every once in a while a book is put together that should be read by every person with a relationship to software development. This book is one of them. Everyone dreams of automating their software testing, but few make it a reality. This down-to-earth book contains stories of 28 teams that went for it, including both successes and failures.

Many books simply provide you the success path. This book also provides you with the steps you could possibly be taking that could lead to failure, helping you to change your path before failing.

The book starts with a nice overview of the case studies and an introduction to the key issues addressed by the case studies. Besides each case study being summarized, it also includes introducing the topics and pointing out the chapter they can be found in. They are broken down into management and technical issues. The management issues include Objectives for Automation, Management Support, Return on Investment and Metrics, Automation in Agile Development, Skills, Planning, Scope, and Expectations, Relationships with Developers, Triggers for Change and Getting Started, Tools and Training, and Political factors.

The technical issues covered include Testware, Abstraction, Architecture, Test Execution Tool, Automation Standards, Reusability, Documentation, Flexibility, Results and Reporting, Testing the Tests, What to Automate, Failure Analysis, and Finding Bugs.

The book includes a really nice table of Case Study Characteristics. Some of the characteristics include location, lifecycle (process used), number of team members, time span, tool types, pilot done, ROI measured, was it successful, and is the project still going on. This table really helps you hunt down topics you are interested in reading about first. The index of this book is really nice also. I mention that because I have seen some books lately where the publisher didn’t want to foot the bill for a nice one. That can be very aggravating.

The book’s last chapter is titled Test Automation Anecdotes. It is filled with experiences from the field that the authors felt were worth repeating, but did not constitute an entire chapter.

The book also has a nice table in the appendix that lists all the tools mentioned in the book. It includes which chapter they are in, where or not they are open source, and a link to the tool owner’s website.

I have repeatedly seen attempts at test automation fail for a verity of reasons. This book included them all: lack of management support, believing the tool is all you need, trying to automate tests without documenting them, and trying to automate every test. The level of difficulty and effort is almost always underestimated. This book definitely puts the level of effort into perspective.

Almost every story’s environment is unique. I really like the way the stories provide solutions to problems that could not be solved by simply purchasing a tool. These solutions are not the industry’s best practice solution, but rather home grown solutions to problems unique to their environment. Now that this book is out they may become best practice solutions. The primary thing they do is make you think out of the box. It is really refreshing to read such a real world book.

Every story is well written and well edited. This is one of the best resources available for helping expand your experience level without having to make the mistakes to learn from along the way. I wish this same format would be done with software projects in general; that is, with the same level of honesty. I see failing projects constantly being touted as successes. Buggy, over budget, late projects, are not a success. Kudos to those authors who stepped up to write about the failed projects!

The authors come from a wide range of technologists. You can check out the 28 case study summaries on the author’s web site.

I only have one word of warning. These stories suck you in. You may find yourself saying “Just one more”, and then suddenly looking at the clock and realizing it is 3 am.

All in all I highly recommend this book to everyone in the business of building software. Before you attempt to automate your testing, read this book! 

Tad Anderson
Tad lives in Mount Joy, PA with his wonderful wife Wendy and his faithful companion Artimus (4 year old Maltese). He has been programming since 1992 and specializes in Software Architecture. He is currently serving in the role of Enterprise Architect at Penn National Insurance.

The Related Post

In order to make the right choices among tools, you must be able to classify them. Otherwise, any choice would be at best haphazard. Without functioning classification, you would not be able to understand new tools fast, nor come up with ideas of using, or creating new tools.
An Overview of Four Methods for Systematic Test Design Strategy Many people test, but few people use the well-known black-box and white-box test design techniques. The technique most used, however, seems to be testing randomly chosen valid values, followed by error guessing, exploratory testing and the like. Could it be that the more systematic test ...
One of the basic challenges with test automation is adoption. I can’t tell you how many times I’ve cataloged licenses for a company and found out they already have many different automation software packages, none of which is being used. Traditionally I’ve been told that is because the tools don’t work and that the teams ...
Introduction A common issue that I come across in projects is the relationship between test automation and programming. In this article I want to highlight some of the differences that I feel exist between the two.
Introduction Many executives have some very basic questions about Software Testing. These questions address the elements of quality (customer satisfaction) and money (spending the least amount of money to prevent future loss). The basic questions that executive have about Software Testing include: Why care about and spend money on testing? Why should testing be treated ...
All too often, software development organizations look at automating software testing as a means of executing existing test cases faster. Very frequently there is no strategic or methodological underpinning to such an effort. The approach is one of running test cases faster is better, which will help to deliver software faster. Even in organizations that ...
When automated tests are well-organized and written with the necessary detail, they can be very efficient and maintainable. But designing automated tests that deal with data can be challenging if you have a lot of data combinations. For example, let’s say we want to simulate a series of 20 customers, along with the number of ...
Test Strategy A test strategy describes how the test effort will reach the quality goals set out by the development team. Sometimes called the test approach, test strategy includes, among other things, the testing objective, methods and techniques of testing and the testing environment.
Jenkins is a Continuous Integration (CI) tool that controls repeatable tasks in software development. Check out this guide to see how TestArchitect seamlessly integrates with Jenkins to establish a CI environment for Automated Testing.
TestArchitect TM is the name we have given to our automation toolset. It reflects the vision that automated testing requires a well-designed architectural plan allowing technical and non-technical elements to work fluidly in their capacity. It also addresses the continual missing link of all test automation tools of how to design tests. In TestArchitect the test ...
Check out the top 12 Automation tools with pros and cons–like Cross-Operating Systems, Cross-Automation Platforms, Programming Language Support, and more – for desktop Automation Testing. Although the demand for desktop app testing is not growing as fast as mobile and web app testing, it’s still a crucial day-to-day duty for many testers, especially those who ...
Introduction In many of the Test Automation projects that we are involved with using our Action-Based Testing methodology, management has expressed a need to relate tests and test results to system requirements. The underlying thought is that automation will create extra possibilities to control the level of compliance to requirements of the system under test. ...

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news