Avoid Epic Fail. Get Professional Help

Bringing in experts can set you up for automation success.

Test automation isn’t easy when your testing gets beyond a few hundred test cases. Lots of brilliant testers and large organizations have, and continue to struggle with test automation, and not for lack of effort. Everyone understands the value of test automation, but few testing organizations realize its full potential.

A year-long IDT (Innovative Defense Technologies) automated software testing survey showed that automated software testing fails to deliver the expected value for the follow reasons:

  • Lack of time: 37%
  • Lack of expertise: 20%
  • Lack of budget: 17%
  • Tool incompatibility: 11%
  • Other (mix of the above, etc.): 15%

Source: Implementing Automated Software Testing by Elfriede Dustin, Thom Garrett and Bernie Gauf

It’s not surprising that lack of time and lack of expertise are cited as the top reasons test automation fails. Test teams often get a tool thrust on them by non-QA decision makers- from a purchasing group that has no idea about testing, or a CIO who was influenced by a tool salesman, their previous company or, something from a conference. These non-QA people do not understand the coding or scripting skill level needed by the team, or may not realize the tool is harder to use than expected, or worse, the tool does not easily work with the application or system under test. As a result , the automation program languishes and then fails. And it’s the test team that looks bad.

If you’re starting a test automation program for the first time, or are charged with making testing faster and better a lot of the challenges will be the same. Test automation success begins at a high level. This is where the lack of (automation) expertise can cripple a project before it gets started.

Many people have unrealistic expectations about how easy or how quickly a program can ramp up and fail to take into account the impact on test schedules. Commitment to time and training and cooperation with developers must come from the highest levels in the organization. Having an outside expert share experiences and educate managers and directors about the complexities and realities of automation programs can set the right expectations and set you up for success by providing:

  • Strategic consulting— assess your current practices and create the strategy for your automation program.
  • Automation Consulting—develop and prioritize the automation effort and supporting activity specific to your needs.
  • Training— instruction in tools, processes and methods.
  • Coaching—hands-on guidance that helps teams become proficient faster.
  • What follows is where organizations can benefit from bringing in a qualified partner to help with their test automation.
Creating the Automation Strategy

Test automation is software development. It needs to be treated like its own project running parallel to production code development, with its own schedule, resourcing and project management. You need commitment and support from the top of the organization for a significant automation suite build-up.

Bringing in someone with prior success in automation implementation—a consultant—to assess your current practices and develop a custom-tailored plan will avoid making costly mistakes. A well-developed strategic plan will define the testing goal and outline the road map to achieve the goal. It will also establish an understanding of testing/quality assurance across the organization, and prescribe staff development to assure the team will have the necessary skills for test automation.

The strategy will guide other aspects of the automation effort, including tool selection. Typically tools are evaluated in this order: cost, ease of use, and features/functionality. There are low-cost easy to use tools— to a point— but getting the tool to do what is needed may require coding/scripting skills the team may not have. Also support is important. Support runs the gamut, from non-existent with open source tools to an included number of hours with commercial tools. The amount of support required will depend on the skills of the team, and it’s good to have when you run into a wall. Also, over-time, you application is likely to undergo quite a bit of change and having a tool that can be customized may be worth paying extra for. These considerations are generally not obvious to teams with little or no prior automation experience and this is where having a well-defined plan will avoid a costly long-term mistake.

Test Automation

This is the stage where the strategy gets implemented, but before just diving in and starting test automation, there’s a lot of work that needs to be done.

You’ll want to build infrastructure that includes naming conventions, standard or common coding practices, source control, error handling, managing data and reporting.

Teams should anticipate spending time to clean up existing tests before automating them. There are significant differences between tests written for manual execution and tests written for automation. Programmers typically don’t pay attention to “designing for testability”. They’re either unaware of how code changes impact automation or don’t effectively communicate changes. The result is that automation is negatively impacted. Failing to address test design issues will result in a large mass of test code that isn’t effective and requires a lot of time just to maintain. When teams understand good test design for automation it will become apparent there will be tests that should be thrown out and replaced with better written tests.

You will also need to develop and prioritize the automation. Taking 100 manual test cases and trying to create 100 automated tests will always be a bad idea. Not all tests are candidates for automation. If the application under test is not stable, it will impact which and how many tests can be automated. Then too, there will be manual tests that can and should be automated, and a lot that shouldn’t be.

Consultants with experience in successful automation implementations can help with all of these areas and enlisting their help at the beginning will get you started on the right track. Once you’re up and running it’s easy to build on success.

Training and Coaching

Training is the essential first step to gaining test automation expertise. There are a lot of good training resources. They range from free, on-demand recordings to generic-instructor led courses, to customized training. We’re seeing more and more companies opt for customized training as it’s not easy to translate generic concepts to specific applications.

The most important training for any team doing test automation, but especially those in Agile and Continuous Integration (CI) environments and teams doing large, high-volume automation, is learning test design. Up-front test design will determine how efficient tests are to run and how easy they are to maintain. Getting this right is what makes it possible to scale automation.

No matter where you are in your automation process, learning low-maintenance test design will enable you to get the most from automation. Action Based Testing (ABT) is a method of test design that lets you do this. It allows you to create common functions and function libraries that make for efficient test design and simplified maintenance. The limitation to training is that it only provides information. You have to turn the knowledge you receive into application, which can be slow, and you’ll probably make mistakes along the way. Getting hands-on instruction will shorten the learning curve dramatically.

Hands-on coaches are the equivalent of personal trainers for testing. Coaches are able to work 1-on-1, and with a small group on your actual testing projects. They can guide you in the test design and the correct approach specific to the application or system under test, and even sit at a keyboard and walk you through a process. Coaches not only help teams become proficient faster, they are also able to jump in and help resolve complex issues so the project keeps moving as expected.

Outsourcing Sense

Automation will not immediately reduce the test effort. In reality the effort typically increases when automation is undertaken. Building the testing infrastructure takes time and utilizes a lot of resources. Designing and creating new tests will also take time. The additional work load can overextend teams already taxed to keep up with current testing demands. So it’s no surprise that lack of time is the number one reason test automation fails.

Outsourcing some or all of the initial work makes a lot of sense. It will be done by people who have done it before. This saves time and avoids mistakes and gets everything integrated and provides you with properly designed tests from the start. Farming the work out can let you can keep up with existing testing, and then have a smooth transition to the new program. When you factor in everything, most of the time this will provide a better ROI than going the in-house route.

Summary

Test automation isn’t easy, but it can be made a lot easier. Expert consultants can assess the structure, staffing, the corporate quality goals, and provide exactly what you need for an effective test automation program. Going this route is not only smart, it will save time, produce long-term benefits, and most of all avoid epic fail.

Joe Luthy
Joe has over 15 years product marketing experience in technology and Internet services for major companies including AT&T and Verisign.TestArchitect for Visual Studio is a keyword authoring platform extension designed specifically to enhance coded UI test automation in Visual Studio 2012.

The Related Post

For this interview, we talked to Greg Wester, Senior Member Technical Staff, Craig Jennings, Senior Director, Quality Engineering and Ritu Ganguly, QE Director at Salesforce. Salesforce.com is a cloud-based enterprise software company specializing in software as a service (SaaS). Best known for its Customer Relationship Management (CRM) product, it was ranked number 27 in Fortune’s 100 ...
Jenkins is a Continuous Integration (CI) tool that controls repeatable tasks in software development. Check out this guide to see how TestArchitect seamlessly integrates with Jenkins to establish a CI environment for Automated Testing.
We’re excited to share with you the latest and greatest features of TestArchitect Gondola, as well as how to use them. So, check them out below! Gondola Studio UI/UX ImprovementsGondola Studio’s new Test Execution Dialog makes it easy to configure and run your test. You can choose the browser or device you’d like to run ...
From automotive Software Testing standards, testing techniques, and process, this article is an in-depth guide for those looking to transfer their existing skills to this exciting industry. For the Software Car, autonomous driving gets most of the hype, but most overlook the fact that there is so much more to Software Testing for the automotive ...
What is Ethereum Smart Contract Testing? What are its challenges? If you’re new to Smart Contract Testing, this in-depth guide will prepare you on how to test smart contracts successfully. Blockchain stands out due to its enormous implications. Everyone has heard of it, but few people know what the ramifications are for testers or how ...
LogiGear Magazine September Test Automation Issue 2017
How lagging automotive design principles adversely affect final products. Cars are integrating more and more software with every model year. The ginormous screen introduced by Tesla in their flagship Model S a few years ago was seemingly unrivaled at the time. Nowadays, screens of this size are not only commonplace in vehicles such as the ...
Two dominant manual testing approaches to the software testing game are scripted and exploratory testing. In the test automation space, we have other approaches. I look at three main contexts for test automation: 1. Code context – e.g. unit testing. 2. System context – e.g. protocol or message level testing. 3. Social context – e.g. ...
I feel like I’ve spent most of my career learning how to write good automated tests in an Agile environment. When I downloaded JUnit in the year 2000 it didn’t take long before I was hooked – unit tests for everything in sight. That gratifying green bar is near-instant feedback that everything is going as ...
Every once in a while a book is put together that should be read by every person with a relationship to software development. This book is one of them. Everyone dreams of automating their software testing, but few make it a reality. This down-to-earth book contains stories of 28 teams that went for it, including ...
LogiGear Magazine – September 2010
Mobile testers need to take a different approach when it comes to Test Automation.

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe