Connecting Test Automation to Specific App Features and Devices

Having the right Test Automation plan helps bridge gaps and fragmentations in the complex mobile environment.

Figuring out the best Test Automation plan is one of the biggest frustrations for today’s digital teams.

Organizations struggle to develop cross-platform Test Automation that can fit with their Continuous Integration cadence, their regression cycles and other elements of their software development lifecycle (SDLC).

On top of the known complexities associated with mobile environments such as the network, the various OS versions, competing apps, there’s also the growing disconnect between the application being tested, the Test Automation scripts and the device.

When an application vendor develops its Android or iOS app, usually the mobile app development team would mark in the property files what features it uses and supports (i.e. GPS, TouchID, Doze, etc.) and what permissions this app requires, such as HTTP connection, GPS and File system.

For the app developer, once these property files are developed, his role in defining the app capabilities is done — which now pushes the test responsibility to the testing teams.

The problem that now arises is, how do these testing teams associate the right test cases based on the app capabilities that are only supported by certain mobile devices?

The screenshot below is taken from a Google Android Developer guide. It’s an example of a line that would usually be in an application Manifest.MF file within the App.APK file. What it’s showing here is that this specific app would require the use of the phone’s camera – this will trigger the user to allow camera access in the app’s first launch post-installation.

While that’s a common case, what would a tester do in a case where the requirement is for TouchID, ForceTouch, Android Doze or Multi-Window (coming in next Android N release)?

These features would complicate the test scenarios and Test Automation because only specific devices and/or OS versions would support these new OS capabilities. Without automatically planning and knowing which test should be executed against which device/OS combination, there is simply no way to manage these tests in today’s mobile app landscape. As a result, many organizations will lean on manual testing as a way to overcome the test delta for newer features that are unique to device/OS combinations. But when this gap grows, test cycles slow down and if not addressed quickly, the balance between manual and automation tests weighs toward manual as more enhancements are introduced in the app.

Break the App-Device-Test Disconnect

The main point here is to continuously make sure that the Test Automation suite supports all app capabilities, and is able to be executed against the right mobile platforms based on all the supported capabilities. So how can you overcome this challenge? Continuing to add to your “manual testing debt” is definitely not a sustainable solution.

Development and testing teams should work together closely, especially when a new supported feature, permission request or other capability is added to the Android app Manifest.MF file (see example to the left) or Apple Info.plist files. Whenever this happens there should be a call to action that requires updating the test execution rules and redirecting existing or newly developed tests for the right devices and platforms.

There are a few ways of addressing this call-to-action:

  • Testing teams receive a new developer build with a change log (pointing to the manifest.mf and info.plist changes)
  • Testing teams validate that there are existing tests for the above changes

⇒ Exists –> OK

⇒ Doesn’t exist –> Implement delta related cases (see the Facebook example above)

  • Testing teams, including developers, are synced on the right target devices (smartphones, tablets, wearables) that match the new and older test cases
  • Schedule test execution with proper rules and redirections to the right devices

In the case of an Android app that supports TouchID within the app Manifest.MF

(uses-permission android:name=”android.permission.USE_FINGERPRINT” /)

the dedicated Test Automation script will only be targeted against Samsung Galaxy S7 as a supported device.

For an iOS application that would use ForceTouch or 3D Touch through the UIApplicationShortcutItems, the developer would typically identify which of the menu options will be exposed to the user for testing to make the test plan easier to frame and develop.

Keeping Test Automation, Apps and Devices Aligned

As mobile operating systems evolve and introduce new functionalities, Test Automation suites are expected to become more fragmented. In order to not lose any existing investment in automation and at the same time increase Test Automation coverage and reduce Manual Testing, it’s important to identify the test suite gaps in your automation and pinpoint the devices that support your unique app features.

By doing this, you’ll be able to synchronize your test elements, your apps being tested and the devices in your test lab.

It’s important to note that “test coverage” will be defined differently based on a company’s industry, its organizational maturity in the digital space and its application types (mobile web, responsive web and/or native apps).

Eran Kinsbruner
Eran Kinsbruner is the mobile evangelist at Perfecto and the author of The Digital Quality Handbook, as well as a monthly columnist at InfoWorld.com. He is a software engineering professional with nearly twenty years of experience at companies such as Matrix, Sun Microsystems, General Electric, Texas Instruments and NeuStar. He holds various industry certifications from ISTQB, CMMI, and others. Eran is a recognized mobile testing influencer and thought leader, as well as an experienced speaker in the major software engineering conferences. He is also a patent-holding inventor for a test exclusion automated mechanism for mobile J2ME testing, public speaker, researcher, and blogger. He can be found all over social media, including on Facebook, Twitter (@ek121268), LinkedIn, and his professional mobile testing blog.

The Related Post

Introduction A characteristic of data warehouse (DW) development is the frequent release of high-quality data for user feedback and acceptance. At the end of each iteration of DW ETLs (Extract-Transform-Load), data tables are expected to be of sufficient quality for the next ETL phase. This objective requires a unique approach to quality assurance methods and ...
Utility: A program that performs a specific task related to the management of computer functions, resources, or files, as password protection, memory management, virus protection, and file compression. Tool: A program or application that software development teams use to create, debug, maintain, or otherwise support other programs and applications. The term usually refers to programs that can be combined together ...
How lagging automotive design principles adversely affect final products. Cars are integrating more and more software with every model year. The ginormous screen introduced by Tesla in their flagship Model S a few years ago was seemingly unrivaled at the time. Nowadays, screens of this size are not only commonplace in vehicles such as the ...
Test automation can provide great benefits to the software testing process and improve the quality of the results…. but its use must be justified and its methods effective. The reasons to automate software testing lie in the pitfalls of manual software testing… As we all know too well, the average manual software testing program:
We’ve scoured the internet to search for videos that provide a wealth of knowledge about Test Automation. We curated this short-list of videos that cover everything from the basics, to the more advanced, and why Test Automation should be part of part of any software development organization. Automation Testing Tutorial for Beginners This tutorial introduces ...
“Testing Applications on the web” – 2nd EditionAuthors: Hung Q. Nguyen, Bob Johnson, Michael HackettPublisher: Wiley; edition (May 16, 2003) This is good book. If you test web apps, you should buy it!, April 20, 2001By Dr. Cem Kaner – Director of Florida Institute of Technology’s Center for Software Testing Education & Research Book Reviews ...
Understanding the benefits and challenges of Automating ERP is critical. According to SAP, ERP (Enterprise Resource Planning) “is the core processes that are needed to run a company: finance, human resources, manufacturing, supply chain, services, procurement, and others. At its most basic level, ERP integrates these processes into a single system. But new ERP systems ...
Even the highest quality organizations have tradeoffs when it comes to their testing coverage. In Japan, Europe, and the United States, automotive manufacturers are aiming to enhance automotive functions by using software; in Japan in particular, Toyota, Nissan, Honda, Mazda, and Subaru are all adding endless amounts of software to their vehicles in the form ...
The huge range of mobile devices used to browse the web now means testing a mobile website before delivery is critical.
Test Strategy A test strategy describes how the test effort will reach the quality goals set out by the development team. Sometimes called the test approach, test strategy includes, among other things, the testing objective, methods and techniques of testing and the testing environment.
When Netflix decided to enter the Android ecosystem, we faced a daunting set of challenges: 1. We wanted to release rapidly (every 6-8 weeks). 2. There were hundreds of Android devices of different shapes, versions, capacities, and specifications which need to playback audio and video. 3. We wanted to keep the team small and happy. ...
There are few topics in quality assurance testing that cause as much confusion as smoke testing versus sanity testing. The two names would seem to describe very different practices— and they do! But people still get them confused, since the distinction is somewhat subtle.

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe