Letter from the Editor – September 2018

As part of my work, I spend a lot of time at client’s sites and talk to various software development organizations. I am beginning to see a problem arise regarding Test Automation. There is too much automation! Surprised?

While there are still many teams struggling to make progress with Test Automation, many teams have been doing very significant Test Automation for a long, long time. Last year, we did a survey on Testing Essentials with our readership to figure out the current trends happening in our field. We discovered that 20% of respondents had no automation at all.

There has always been a lot of focus on organizations who are just starting on their Test Automation journey. There is good reason for this. They need the most help.

But, often times I think too little focus is given to companies that have been automating for a while who have big automation suites. As these automation programs continue to grow and mature, they need a different kind of help. In light of businesses trying to deliver faster than ever through Continuous Delivery and, even Continuous Deployment, long time automating businesses with overly large regression suites are possibly hindering their organization’s progress.

Continuous Delivery demands immediate feedback. When a team has been automating for a while, has a decent framework and some smart people; thousands of tests are easily attainable. When you run them against multiple VMs, configurations, devices or browsers, it can take days to get feedback. But, when the pipeline is automated, you think can run the full regression suite and let the team know what happens in 5 days? No.

This is a very common issue today that clearly can not continue. So what is a team supposed to do? The solutions are difficult, if not painful. We need to cut tests, clean up and clean out tests; remove duplicates in order to reduce the number of tests for faster runs. Cleaning out old, failing tests and duplicates is an easy idea but may take a long time to do. There’s a catch, cutting seemingly important tests to get faster runs but cutting coverage may be a political battle. We need to trace and tag tests to code and functions to streamline runs. But, cutting coverage increases risk. Team members and managers do not want to hear this.

The most important thing to remember at this point is that CD’s goal is for smaller changes with smaller releases. The idea is to roll out one release at a time, to keep it isolated, thereby limiting risk. Our automated suites must reflect this. It’s easy to visualize this idea using containers. If you swap out one container of code, do you have to run your 5 day full regression suite? Also remember, CD is based on significant, repeatable, formalized re-running of developer unit tests. The typically larger increase in unit level tests can offset the increased risk of cutting API or UI level tests.

Any company moving to CD knows there are new risks with speedier deployments. Risk should be well-known and communicated. Risk should be managed. Risk should be localized and small in scope, with easy roll-backs. There is a give and take here. It’s not only faster and faster, smaller deploys mean less risk. When I see an organization’s automation goal of “More, more, more!”, I tell them a better goal is to stay Lean and Mean.

Our Test Automation issue is one of the biggest and best read every year. That is a good thing. I hope there is useful information about Test Automation for every team- teams new to automation as well as long time automators. New ideas and help is a good thing for all of us.

In this issue we look in-depth at automation across a wide variety of spectrums. Our own Tu Nguyen and Christine Paras discuss How to Automate Oracle ERP Testing in the cover story. James Willett discusses Alan Richardson’s book, Automating & Testing a REST API, and Daniel Knott returns to discuss the mobile test pyramid. This issue of TestArchitect Corner discusses how TestArchitect integrates with HP Quality Center. Rounding up the lineup, Eran Kinsbruner discusses how to connect Test Automation to specific app features and devices. This issue is also filled with a resource section on automation videos for you to watch, and is great for newbies and advanced practitioners alike. We also have an infographic on Anti-Patterns and how to avoid them. Don’t forget to peruse our events Calendar. LogiGear will be at both STARWEST and STPCon-Fall. If you’re planning on attending these shows then stop by and say hello!

Michael Hackett
Michael is a co-founder of LogiGear Corporation, and has over two decades of experience in software engineering in banking, securities, healthcare and consumer electronics. Michael is a Certified Scrum Master and has co-authored two books on software testing. Testing Applications on the Web: Test Planning for Mobile and Internet-Based Systems (Wiley, 2nd ed. 2003), and Global Software Test Automation (Happy About Publishing, 2006). He is a founding member of the Board of Advisors at the University of California Berkeley Extension and has taught for the Certificate in Software Quality Engineering and Management at the University of California Santa Cruz Extension. As a member of IEEE, his training courses have brought Silicon Valley testing expertise to over 16 countries. Michael holds a Bachelor of Science in Engineering from Carnegie Mellon University.

The Related Post

I have been training testers for about 15 years in universities, corporations, online, and individually – in both a training, managing and coaching capacity. So far, I have executed these various training efforts in 16 countries, under good and rough conditions – from simultaneous translation, to video broadcast to multiple sites, to group games with ...
I led the Editor’s Note in our very first mobile issue with “Everything is mobile”, but it is now way beyond what we thought. Mobile has come to mean only the smart phone, mobility is the word that describes everything a smart phone enables you to do. Mobility is more than a device! Mobility is ...
Test automation is a big topic. There are so many different areas to talk about: tool choice, jumpstart, cross platform, services, cloud… Each of these areas have changed so much in the recent past that they could each be worth their own magazine issue.
On the whole, everyone wants to do a great job, have a better work environment, happy clients and customers, and to be employed by a company earning lots of money. All great goals! But this is not always the case. When it is not, you can suggest process improvements, better tool use, different estimating techniques, ...
Big and complex testing. What do these terms conjure up in your mind? When we added this topic to the editorial calendar, I had the notion that we might illustrate some large or complex systems and explore some of the test and quality challenges they present. We might have an article on: building and testing ...
I once consulted for a company to give a week-long course on testing and QA. It was a survey course covering a wide range of topics. I was setting up and chatting with students in the room. One man came over to me and said: “I have been testing for 6 months and I am completely ...
As fast as Mobile is growing, the platform is still immature and is evolving at a very rapid pace. While there are whole countries that have migrated large government services to mobile, countries ranging from Estonia to Turkey to Kenya have many longtime mobile users have yet to use mPay or other mobile payment systems. ...
For everyone still celebrating holidays: Happy Lunar New Year! At this time of the year many teams and companies are starting new projects, new initiatives, and hiring new staff. LogiGear Magazine will continue to be the resource for you for better testing with much less stress! We are excited about the focus of this month’s ...
As we settle into autumn, we’re taking the time to start some new traditions. This is LogiGear magazine’s first issue on SMAC. SMAC—social, mobile, analytics and cloud. We will be doing more issues in the next few years on these topics since so much of the product world is moving to this development stack.
The Greek philosopher Heraclitus of Ephesus (c. 500 BCE) is credited with saying, “The only constant is change.”   This is a statement that, more than 2,000 years later, still holds true. Today, we are in a time of great change. Everything is in flux. The fact is, we are always in a state of change even if ...
There is a growing software development dynamic of teams without Testers. When I first went into Software Quality, I learned one thing right away: My role was user advocate. My main job was to find bugs. This is the Lean principle called Amplified Learning. We learn about behavior by testing. Even then, validation was not ...
Continuous Testing… what is it? When we first decided to do a magazine issue dedicated to the DevOps practice of Continuous Testing, I joked with someone: “It’s about testing continuously.” And their reply was: “Yeah. What else would it be?” I was joking, but clearly the joke didn’t land. Continuous Testing is about testing continuously, ...

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe