The Automation of Mobile Testing – Should We or Shouldn’t We . . .

6.3 -Sep-2014-03Manual testing teams may not be able to test all the processes with each build

Test automation of applications has been around for many years. There are many of us in the automated testing field that started very early in the test automation phase, but the introduction of mobile devices has brought on a new angle in test automation of applications. Mobile device testing introduces new technical challenges along with new processes and testing tools. But should we use test automation for mobile applications that are on mobile devices? Many of us have mobile devices and we get our applications from the stores on Android or iOS and we go about our business.

Quality, performance and business rules?

As we know, there are at least three kinds of applications for Mobile Devices; Native, Mobile Web, and Hybrid. Like any application, developers need to build it, and testers test it. Now, when we introduce methodology into the mix, we add challenges. This agile development process introduces tight timelines and tight delivery dates. Developers are challenged to get the business processes into the applications as quickly as possible. This adds multiple builds daily, sometimes hourly. In the desktop and standard web testing process, whether automated or manual, we would expect to regression test each build, right? How many testers would you need to do that when builds are being delivered so quickly? Can manual testers do that and keep up with what they have tested, along with making sure that the business processes have been tested each time?

When there is a time crunch, usually there will be decisions made to stick to the critical business processes or new functionality and leave it at that. It sounds great, but is that the best way to test and make sure a quality product has been delivered to production? Users are accustomed to application pages being rendered within a few seconds, which means response times need to be verified before going into production. Users are accustomed to the application functionality being consistent each and every build. If there is a new build, users are not expecting to lose functionality or any visual effects. Our eyes are accustomed to application pages looking great each and every build. How is a small test team going to do that? Honestly, a manual testing team will probably not be able to test all the processes each time and will revert to the 80/ 20 rule, or something close. Now, there are some teams out there that will claim to have an army of testers that they will call up and have the start testing the build. That is awesome, however, even that army of testers may take the common path or the happy path. They may not have all the business rules or the defects to look at to see what the challenges or critical areas of the app are. How do they report back to a central point and collaborate on what they have tested and what the results were? How would they also add requirements trace-ability into the process? Unless this is a seasoned group, that will be a challenge.

Where test automation comes in.

From wherever you are in the world, using test automation can make testing more complete and add requirements coverage, test coverage, trace-ability, defect tracking, and overall test reporting easier to manage. From one report, management can see what percentage of the application has been tested, what defects have been introduced and what defects have been fixed. With test automation, when a defect is found, it is very easy to recreate that defect and the steps for it. When manual testers test and find a defect, if the steps are complex enough, then chances are, the tester will not follow the same path and the defect may not be reproduced. That may be a big issue.

With test automation, test scripts can be run off hours if needed. Now with mobile device testing, that will add complexity because some environments and devices will need to be configured to run the test. That may or may not be able to be automated. Either way, the scripts can be run and tests can be run without manual intervention once set up. With test automation, when builds are done multiple times during the day, we can find the changes to the application much faster than a manual tester.

How fast can scripts do that?

Well, that would depend upon the environments being used, the complexity of the application under test, and the performance of the application.

For the most part, running automated scripts will always test more functionality than a manual tester. If there is a critical path that needs to be tested, the scripts can be run specifically for that and they will be executed the same each and every time. For many teams, cost comes into the picture. Most departments have budgets and test automation tools are usually pretty expensive. Once research is done, it will be found that there are many tools out there that can create scripts for a fairly inexpensive cost with even cheaper renewal costs. For the teams that do not have much experience with test automation, it would be better to try a small project with one of the less expensive tools. And remember, cheaper cost does not mean cheaper in the way the tool works, but it may mean that you don’t get all the bells and whistles. Compromises will always need to be added.

Should you use test automation for mobile device testing?

We can ask the same question for the test automation of a desktop application. It does not just need to be answered for mobile device testing. If your team plans to create subsequent updates to the app, or create the application in phases, I would highly recommend some sort of test script automation. If the plan is to just create a quick application one time and that is all, then it is up to the team to decide whether or not it is cost effective to automate the testing process. There will always be a benefit to test automation as a whole because test automation is not just script execution. Test automation is the whole testing process. That is where the real benefit for the whole project comes in. But for development and testers, script automation will help by allowing the manual testers to move onto some other critical tasks and leave the regression to the automation team.

Whether you have 4 or 400 testers, there will always be something that will be missed. Adding test automation will add test coverage and higher requirements coverage to your testing results. Mobile Device test automation will use a lot of the same processes that standard test automation uses. The tools may be different, but the rest will be similar.

(This article was originally published on www.olenick.com/blog/automation-mobile-testing/)

 

Sam Mullin

Sam Mullin is a specialist in Mobile Device Test Automation. He has contributed to test planning, design, and execution of automated testing efforts within the healthcare, electricity, and banking industries. He continues to work on providing specialized support for mobile device test automation using various mobile device testing tools. Currently, Sam is working for Olenick and Associates located in Chicago Illinois (www.olenick.com)

Sam Mullin
Sam Mullin is a specialist in Mobile Device Test Automation. He has contributed to test planning, design, and execution of automated testing efforts within the healthcare, electricity, and banking industries. He continues to work on providing specialized support for mobile device test automation using various mobile device testing tools. Currently, Sam is working for Olenick and Associates located in Chicago Illinois

The Related Post

To ensure the success of an app, QA must be involved in all stages of development. Quality Assurance (QA) plays a vital role in the development of mobile applications, but many overlook the critical nature of this piece of the app development process. To ensure the success of an app, QA must be involved in ...
LogiGear Magazine, September 2015: Mobile Testing
I’ve spent the last six months or so testing mobile apps for both iOS and Android. Here’s eight of my key lessons learned: Automated UI testing tools for mobile apps are immature: whilst tools like WebDriver for automated UI testing of web apps are very mature, automated UI testing of native mobile apps is the ...
What you need to know to get started Introduction to mobile application testing Gone are the days when the telephone was an appliance that sat in a corner and had to ring to get our attention, or a computer was a machine only few people used — these devices are now an extension of our ...
The most significant facet of mobile testing is understanding the mobile ecosystem. Mobile applications are growing in use and sophistication along with the speed of the networks and the increasing power of the devices. To be sure, mobile application testing can be done with minimal automation, but a high level of test automation will help ...
iOS culture, even in many large organizations with skilled engineers, is behind on up-to-date testing practices. Agile development has long been all the rage; indeed, in most modern development shops the great agile methodologies are old hat. If you come from a software background like Ruby on Rails, Python, or certain Java niches, you may–until ...
Great mobile testing requires creativity to find problems that matter. I’d like to take you through the thought process of testers and discuss the types of things they consider when testing a mobile app. The intention here is to highlight their thought processes and to show the coverage and depth that testers often go to.
This is the second part of a two part article that analyzes the impact of product development for the internet of things (IoT) on software testing.  Part one of this article (LogiGear Magazine, Sept 2014) gave a wide view on the IoT, embedded systems, and the device development aspects of testing on these projects. This ...
Organizations need to implement automated testing strategies designed specifically for mobile applications. Mobile device usage continues to gain momentum at increasing speed. Enterprises that delay mobile adoption face the danger of becoming competitively disadvantaged. But, before jumping in headlong, you need to be fully aware of the unique challenges that can arise when developing and implementing ...
I am not a big fan of concepts which moves industry standards to IT. I am rather a Agile and Scrum guy. Managing multiple projects at once and trying to set a highest quality standard is a challenge and this book shows how industrial language can be translated into software development. I do not think that it ...
To start with, we need a Test schedule. The same is created in the process of developing the Test plan. In this schedule, we have to estimate the time required for testing of the entire Data Warehouse system. There are different methodologies available to create a Test schedule. None of them are perfect because the ...
A fully integrated test automation platform for mobile testing Mobile application testing is taking on greater significance as just about everything done on the desktop is transitioning to mobile. It’s taken a while for mobile testing in the consumer space to be made much of a priority. Now that sensitive data is being passed back ...

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe