Letter from the Editor – September 2018

As part of my work, I spend a lot of time at client’s sites and talk to various software development organizations. I am beginning to see a problem arise regarding Test Automation. There is too much automation! Surprised?

While there are still many teams struggling to make progress with Test Automation, many teams have been doing very significant Test Automation for a long, long time. Last year, we did a survey on Testing Essentials with our readership to figure out the current trends happening in our field. We discovered that 20% of respondents had no automation at all.

There has always been a lot of focus on organizations who are just starting on their Test Automation journey. There is good reason for this. They need the most help.

But, often times I think too little focus is given to companies that have been automating for a while who have big automation suites. As these automation programs continue to grow and mature, they need a different kind of help. In light of businesses trying to deliver faster than ever through Continuous Delivery and, even Continuous Deployment, long time automating businesses with overly large regression suites are possibly hindering their organization’s progress.

Continuous Delivery demands immediate feedback. When a team has been automating for a while, has a decent framework and some smart people; thousands of tests are easily attainable. When you run them against multiple VMs, configurations, devices or browsers, it can take days to get feedback. But, when the pipeline is automated, you think can run the full regression suite and let the team know what happens in 5 days? No.

This is a very common issue today that clearly can not continue. So what is a team supposed to do? The solutions are difficult, if not painful. We need to cut tests, clean up and clean out tests; remove duplicates in order to reduce the number of tests for faster runs. Cleaning out old, failing tests and duplicates is an easy idea but may take a long time to do. There’s a catch, cutting seemingly important tests to get faster runs but cutting coverage may be a political battle. We need to trace and tag tests to code and functions to streamline runs. But, cutting coverage increases risk. Team members and managers do not want to hear this.

The most important thing to remember at this point is that CD’s goal is for smaller changes with smaller releases. The idea is to roll out one release at a time, to keep it isolated, thereby limiting risk. Our automated suites must reflect this. It’s easy to visualize this idea using containers. If you swap out one container of code, do you have to run your 5 day full regression suite? Also remember, CD is based on significant, repeatable, formalized re-running of developer unit tests. The typically larger increase in unit level tests can offset the increased risk of cutting API or UI level tests.

Any company moving to CD knows there are new risks with speedier deployments. Risk should be well-known and communicated. Risk should be managed. Risk should be localized and small in scope, with easy roll-backs. There is a give and take here. It’s not only faster and faster, smaller deploys mean less risk. When I see an organization’s automation goal of “More, more, more!”, I tell them a better goal is to stay Lean and Mean.

Our Test Automation issue is one of the biggest and best read every year. That is a good thing. I hope there is useful information about Test Automation for every team- teams new to automation as well as long time automators. New ideas and help is a good thing for all of us.

In this issue we look in-depth at automation across a wide variety of spectrums. Our own Tu Nguyen and Christine Paras discuss How to Automate Oracle ERP Testing in the cover story. James Willett discusses Alan Richardson’s book, Automating & Testing a REST API, and Daniel Knott returns to discuss the mobile test pyramid. This issue of TestArchitect Corner discusses how TestArchitect integrates with HP Quality Center. Rounding up the lineup, Eran Kinsbruner discusses how to connect Test Automation to specific app features and devices. This issue is also filled with a resource section on automation videos for you to watch, and is great for newbies and advanced practitioners alike. We also have an infographic on Anti-Patterns and how to avoid them. Don’t forget to peruse our events Calendar. LogiGear will be at both STARWEST and STPCon-Fall. If you’re planning on attending these shows then stop by and say hello!

Michael Hackett
Michael is a co-founder of LogiGear Corporation, and has over two decades of experience in software engineering in banking, securities, healthcare and consumer electronics. Michael is a Certified Scrum Master and has co-authored two books on software testing. Testing Applications on the Web: Test Planning for Mobile and Internet-Based Systems (Wiley, 2nd ed. 2003), and Global Software Test Automation (Happy About Publishing, 2006). He is a founding member of the Board of Advisors at the University of California Berkeley Extension and has taught for the Certificate in Software Quality Engineering and Management at the University of California Santa Cruz Extension. As a member of IEEE, his training courses have brought Silicon Valley testing expertise to over 16 countries. Michael holds a Bachelor of Science in Engineering from Carnegie Mellon University.

The Related Post

A while ago, I helped start a Software Quality Certificate Program as a part of the Software Engineering Program at the University of California, Santa Cruz Extension in Silicon Valley. I was on the Board of Advisors. While putting the curriculum together, a few people suggested a Measurement and Metrics course. Since I was teaching ...
If you are reading this issue, you are probably aware of the impact on the business world of cloud computing. Most people do not have a good grasp on what the cloud is or how people and products can use it. BTW, you are already a cloud user. If your email is stored somewhere “on ...
Happy New Year from LogiGear to those of us who celebrated New Years on January 1! And for our lunar calendar followers, an almost Happy New Year come February 3rd. We look forward to an exciting and full 2011 as its predecessor was a tough year for many in the software business. At LogiGear Magazine, ...
I once consulted for a company to give a week-long course on testing and QA. It was a survey course covering a wide range of topics. I was setting up and chatting with students in the room. One man came over to me and said: “I have been testing for 6 months and I am completely ...
Testing Embedded systems and testing the Internet of Things could each have their own issue of LogiGear magazine. But these days they are referred to presupposing knowledge of the other, so we thought it would be a good idea to tackle the two together in this issue to give a broad understanding of the landscape ...
There has been a tectonic shift in software development tools in just the past few years. Agile practices and increasingly distributed teams have been significant factors but, in my opinion, the main reason is a new and more intense focus on tools for testing driven by more complex software and shorter development cycles. There have ...
This is our third issue concerning topics of Continuous Delivery (CD) and DevOps with the inclusion of Continuous Testing. DevOps has been around for a while and I hope the period of buzz is over and companies moving towards building a development pipeline have begun their process, including changing their test strategies.
Our plan for the December LogiGear Magazine was to have a forward-looking Trends and Challenges issue. However, whilst assembling our September issue on SMAC, we realized the momentum SMAC was gaining in the industry. We had a large amount of content on our hands from a range of excellent contributors. Thus, we decided to split ...
Software development projects are multifaceted. There is staffing and budget work. There are communication and team dynamics. There are project and process issues from what the customer wants, when they want it, revenue projections, and production dates. As part of my work in helping people deliver software, I get involved in all aspects mentioned above. ...
Testers need to learn their craft and hone in on their skill set. That means building skills, sharpening their tools, and becoming creative detectives. There is no cookie-cutter tester and no best practice. The best circumstance is a fully-skilled, aggressive tester mixed with curiosity, nimbleness, and agility.
The Greek philosopher Heraclitus of Ephesus (c. 500 BCE) is credited with saying, “The only constant is change.”   This is a statement that, more than 2,000 years later, still holds true. Today, we are in a time of great change. Everything is in flux. The fact is, we are always in a state of change even if ...
For everyone still celebrating holidays: Happy Lunar New Year! At this time of the year many teams and companies are starting new projects, new initiatives, and hiring new staff. LogiGear Magazine will continue to be the resource for you for better testing with much less stress! We are excited about the focus of this month’s ...

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe