The Midpoint Crisis: How Automation Can Make More Manual Testing Work

Picture a series of sprints: There are a variety of features being developed, with an eye towards having automated tests related to those features. Work starts to move along and Test Automation work likewise goes along with it. However, at some point, there invariably is that moment, usually in the middle of the project, where the need to do End-to-End Testing or some other aspect comes into play and there is now a need for a great deal of Manual Testing. Why? We have run into that specter I like to refer to as “The Midpoint Wall” or “The Midpoint Crisis.” 

In short, if I am not careful or attentive at various stages, even with my best efforts, I can find myself in a situation where even having done my best and most vigilant work, I can be surrounded by a list of manual steps that now have to be performed. Have I been here? Certainly. The question, though, should be: “Why do I find myself here so frequently?” 

What is a “Midpoint Crisis”?

First, before you furiously Google “Midpoint Crisis” to find out what I’m talking about, let me define it. You won’t find a definition for the “Midpoint Crisis” as it’s a term I (mostly) made up. However, it’s very similar to Steven Covey’s 7th Habit of “Sharpening the Saw.” 

“Sharpening the Saw” essentially means to seek continuous improvement and renewal professionally and personally. How does that fit into the idea of the “Midpoint Crisis”? It’s the point where the odds of the idea of stepping back and sharpening the saw are the least appealing.

Here’s a mental image for you: You have a log that you want to saw. You have been at it for a while now, and you are about halfway through the log. You keep going when someone comes by and this is how the conversation goes:

Them: “How long have you been at this?” 

You: “Hours!” 

Them: “Wow, that’s a long time. What if you sharpen the saw?”

You: “I can’t do that silly, I’m too busy sawing!”

That’s the essence of the Midpoint Crisis: We are having to choose with doggedly pushing through manually or focusing on regression when, if we take the time to “sharpen the saw,” we might be able to do more and more effectively. Unfortunately, it’s hard to see that when we are at the midpoint of a project.

Now that you have a better idea of what a “Midpoint Crisis” is, let’s look at some common culprits that cause Midpoint Crises, and then see how we can manage through them.

Culprit 1: Testability

One of the areas that I find most frequently that leads to a Midpoint Crisis is the discovery of an area that makes for a difficult situation in regards to testability. One area that could be particularly frustrating is if you are dealing with dynamic data and, due to that data being changeable, you have elements that are generated at random. If I was aware of the fact that these elements would have been created randomly, I might have asked to see if there was another way that I could reach and set/unset or query them. Perhaps it is a process that is deeply nested in a variety of steps and menu transitions, or the information I need to get to requires traversing iFrames. Perhaps there is a level of necessary cross-server scripting that has to be anticipated. 

I have found myself in this situation many times and ultimately it comes down to not knowing enough about the system or the necessary interactions before going in. The recommendation of course is to have a conversation with the team early on and determine what the testability options are and how we might reach them. To put it simply, we need to have this conversation sooner rather than later. The best time to talk about this is during the design workshop. If, however, you are in the middle of automating and you find that you have to switch gears because your automated tests will not work or the feature in question is resistant to Automation, the next best time to have that conversation is “now.”

As an example, I recently tried (with an emphasis on tried) to go in and attempt to navigate through a server and examine/set values needed in a deeply nested process. It required identifying a customer, finding their profile information, getting to their extended settings options, and then clicking through a variety of listings to find the right values and then enter them. Using a process, like WebDriver, made for long and drawn out interactions that were prone to breaking midway through the tests. I brought this up to the Developer and they said, “It’s not our server specifically so we have limited control over how to interact with it. However, I might be able to set up some code services that you can use and set those values directly through an API rather than this drawn out process.” With that, I was able to switch away from an extensive WebDriver Automation attempt (and frequent Manual Testing to fill in the gaps) by taking an alternate path. It’s important to recognize that that alternate path won’t be made unless we as Testers bring up the testability of what we are working with––don’t be afraid to ask!

Culprit 2: The End-to-End Maze

In simpler times, we could be expected to work on a standalone product and that product would have limited interactions with other systems. Unfortunately, for the most part, that is not the reality of many Testers today. Today’s systems are complex, have many moving parts, and as highlighted in my previous section, may require us interacting with systems and data that don’t originate with us or that are not necessarily under our control. Still, I have a need to test to make sure everything works and that often causes issues with Test Automation. Sure, I can automate my own areas but what do I do with those downstream systems? Can I reach them? Do I have access to interact with them? Can I change the data on their end to demonstrate the effect on my area of testing concern?

These are typical areas that I find to be both frustrating and inevitable. This can also be compounded when dealing with multiple systems set up in the cloud. When we have a system we are testing that requires interaction with other microservices hosted on other servers, and those other servers are part of different groups, the logistics often end up looking like a military campaign.

While mocking and stubs are helpful when it comes to development and doing unit tests, these will not help us when it is time to deploy and test these systems for real and with real data (not to mention at scale). This is where I encourage Testers, myself included, to develop what I like to call a “sphere of influence” map. This is a process where I diagram out all of the possible end-to-end interactions and with that, the servers/services I need to interact with. I reference this document in meetings and make sure that, if a new feature gets added, that I am clear if the feature touches on something within my sphere of influence. More importantly, I need to identify when it doesn’t. If a new microservice has been enabled on a new docker container set up on an EC2 server somewhere, I need to know if I can reach it and if my credentials are going to work. It is frustrating to find myself in the middle of work and having to ferret out what services are allowing me to interact and which ones aren’t and why. By identifying these early and doing the necessary diligence to make sure I can get to each service and interact, I can avoid a bottleneck later.

Culprit 3: Geography

How often have I worked on a test, manually walking through everything to make sure I have my workflow steps repeatable; yet, found that when automating, my tests aren’t working due to latency issues? This can be a real issue when it comes to geographically diverse regions and companies that hire people that work in a distributed manner. 

Generally speaking, I write tests to work on my local system first, and then I run those tests on a variety of environments (we have QA, Staging, and Production machines set up for these scenarios). The challenge comes if I have to run from my own machine in my home office, connect to machines across the country (or in some cases across an ocean), and expect the tests to behave appropriately. Too often, they don’t. This can also be exacerbated with a hybrid cloud arrangement, where some services are in the cloud and other services are hosted by my company. 

Over time, we have made this process cleaner by keeping our systems and their dependencies as geographically close as is practical. We have offices and customers all over the world and as such, that has meant that we have had to plan out clusters of machines in a variety of geographical areas to allow all theatres of operation the best performance possible. Likewise, when it comes to testing, if I have a geography issue, then I need to overcome that geographical challenge. It may work out that my development environment is only a first step and then all of my tests need to be packaged up and set up on a test server located in the geographical area where the servers are located. Likewise, my company has also made it possible to have servers in the cloud that are geographically located close to where I am so that when I am developing my tests, I do not have to deal with these geographical limitations as frequently. Once I am satisfied that my tests work, then I have the ability to export them to servers in different regions and run them there.

Culprit 4: Limitations to the Framework

I have been both fortunate enough to have a system that is well worked out and unfortunate enough to run into areas where nothing exists when it comes to new feature Automation work. Given time and attention, we can build out the needed Automation libraries to handle the new features and issues, but if we have that gap as a deadline is looming, then we are stuck with having to choose to automate or test. Sometimes, we just don’t get the option to do both. 

In my company, we have automated tests and a code review for a feature to be considered “done” on the Scrum Board. Have I had to carry stories over to the next sprint because I couldn’t get it done in time? Sure. The good news is that having done that a number of times, it has allowed my team the opportunity to look and see what is missing. At times, I am able to fill in the gaps, but sometimes we need someone with more experience to come in (occasionally from another team if necessary) to help me get over that particular hurdle. 

For those familiar with the “7 Habits” philosophies, this is the “Sharpen the Saw” step. Yes, it can be difficult to do the necessary work in the woodshed and mind the store at the same time. Still, my having these conversations to address the gaps help make these necessary steps achievable.   

Conclusion

While it is not always possible to strike the perfect balance between Manual and Automated Testing, it helps to look at the various culprits listed above that can make my job more difficult than it needs to be. I want to emphasize that by no means is Manual Testing bad; there are still many great uses for Manual Testing like ad hoc or Exploratory Testing. However, the issue in regards to the Midpoint Crisis is that often these crises occur at times when ample time and resources haven’t been allotted to successfully complete the now-required Manual Testing (thus, the Midpoint Crisis). It would be great if we could always have enough time and bandwidth to go back and “sharpen our saw” in regards to our Automation, but you and I both know that’s not always possible; so, it’s important that we have the foresight to recognize possible crisis causes before we begin. By focusing on testability, an end-to-end strategy that makes sense, paying attention to how our infrastructure is configured, and occasionally getting help from others, we can keep these Midpoint Crises to a more manageable level. 

Michael Larsen
Michael Larsen is a Senior Automation Engineer with LTG/PeopleFluent. Over the past three decades, he has been involved in software testing for a range of products and industries, including network routers & switches, virtual machines, capacitance touch devices, video games, and client/server, distributed database & web applications.

Michael is a Black Belt in the Miagi-Do School of Software Testing, helped start and facilitate the Americas chapter of Weekend Testing, is a former Chair of the Education Special Interest Group with the Association for Software Testing (AST), a lead instructor of the Black Box Software Testing courses through AST, and former Board Member and President of AST. Michael writes the TESTHEAD blog and can be found on Twitter at @mkltesthead. A list of books, articles, papers, and presentations can be seen at http://www.linkedin.com/in/mkltesthead.

The Related Post

The 12 Do’s and Don’ts of Test Automation When I started my career as a Software Tester a decade ago, Test Automation was viewed with some skepticism.
The huge range of mobile devices used to browse the web now means testing a mobile website before delivery is critical.
LogiGear Magazine, December 2015: Test Automation
As I wrote in various articles, organization is one of the 3 key requisites for successful automated testing, the other two being test design and automation architecture.
The Cloud demands that we be as nimble as possible, delivering features and fixes in almost real-time fashion. Both customer and provider rely on software development that can maintain quality while being light on its feet and constantly moving. In addition, Cloud-oriented systems tend to be highly complex and dynamic in structure — more than ...
People who know me and my work probably know my emphasis on good test design for successful test automation. I have written about this in “Key Success Factors for Keyword Driven Testing“. In the Action Based Testing (ABT) method that I have pioneered over the years it is an essential element for success. However, agreeing ...
What is the Automation ROI ticker? The LogiGear Automation Return on Investment (ROI) ticker, the set of colored numbers that you see above the page, shows how much money we presumably save our customers over time by employing test automation as compared to doing those same tests manually, both at the design and execution level.
When automated tests are well-organized and written with the necessary detail, they can be very efficient and maintainable. But designing automated tests that deal with data can be challenging if you have a lot of data combinations. For example, let’s say we want to simulate a series of 20 customers, along with the number of ...
From cross-device testing, to regression testing, to load testing, to data-driven testing, check out the types of testing that are suitable for Test Automation. Scene: Interior QA Department. Engineering is preparing for a final product launch with a deadline that is 12 weeks away. In 6 weeks, there will be a 1 week quality gate, ...
As our world continues its digital transformation with excitement in the advancement and convergence of so many technologies- from AI, machine learning, big data and analytics, to device mesh connectivity, nor should we forget VR and AR- 2017 promises to be a year that further transforms the way we work, play and take care of ...
We’re excited to share with you the latest and greatest features of TestArchitect Gondola, as well as how to use them. So, check them out below! Gondola Studio UI/UX ImprovementsGondola Studio’s new Test Execution Dialog makes it easy to configure and run your test. You can choose the browser or device you’d like to run ...
Investing in Test Automation training will increase your team’s productivity. The availability of reliable jobs in a competitive US market seems to be constantly embattled with competition and replacements of artificial intelligence (AI). In 2016, Foxconn replaced 60,000 employees with robots. However, the growth of Test Automation as an occupation has highlighted an intriguing option ...

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe