A Beginner’s Guide to Mobile Application Testing


What you need to know to get started

Introduction to mobile application testing

Gone are the days when the telephone was an appliance that sat in a corner and had to ring to get our attention, or a computer was a machine only few people used — these devices are now an extension of our being — a window to the world and virtual servants that do as they are told. Computers became a rage and changed how we humans thought, behaved, learned and existed.

Mobile technology and smart devices are the trend now and will change the future of the world as we know it. We all can vouch for that, can’t we?

Now, it would be amateurish if I were to list what we use these mobile devices for. You all know it — perhaps better than I do.

So let’s get straight to what this tutorial is all about:

This tutorial will serve as both an introduction, and your guide, to mobile testing. So, read through!

Types of mobile testing

Broadly speaking, there are two kinds of testing that take place on mobile devices:

Hardware testing

Testing of the device, including the internal processors, internal hardware, screen sizes, resolution, space or memory, camera, radio, Bluetooth, WIFI, etc. This is sometimes referred to as, simply, mobile testing.

Software or Application testing:

The applications that work on mobile devices, and their functionality, are tested. This is called mobile application testing to differentiate it from the earlier method. And among mobile applications, there are a few basic differences that are important to understand:

Native apps: A native application is one that is created for use on a platform like mobile and tablets.

Mobile web apps: These are server-side apps to access website(s) on mobile using different browsers, like Chrome and Firefox, by connecting to a mobile network or wireless network like WIFI.

Hybrid apps: Combinations of native app and web app. They run on devices or offline and are written using web technologies like HTML5 and CSS.

There are a few basic differences that set these app types apart:

  • Native apps have single-platform affinity while mobile web apps have cross-platform affinity.
  • Native apps are written in platforms like SDKs while mobile web apps are written with web technologies like HTML, CSS, ASP.NET, Java, and PHP.
  • For a native app, installation is required; for mobile web apps, no installation is required.
  • Native apps can be updated from the play store or app store while mobile web apps are centralized updates.
  • Many native apps don’t require an Internet connection, but for mobile web apps it’s a must.
  • In general, native apps works faster than mobile web apps.
  • Native apps are installed from app stores like Google Play Store or the Apple app store while mobile web apps are websites and are only accessible through the Internet.

From here on, the rest of this article is about mobile application testing.

Significance of mobile application testing

Testing applications on mobile devices is more
challenging than testing web apps on desktops. This is due to:

  • A different range of mobile devices with different screen sizes and hardware configurations like hard keypad, virtual keypad (touch screen) and trackball, etc.
  • Wide varieties of mobile devices like HTC, Samsung, Apple and Nokia.
  • Different mobile operating systems like Android, Symbian, Windows, Blackberry and IOS.
  • Different versions of operation systems like iOS 5.x, iOS 6.x, BB5.x, BB6.x, etc.
  • Different mobile network technologies like GSM and CDMA.
  • Frequent updates — like Android 4.2, 4.3, 4.4; or iOS-5.x, 6.x — and with each update a new testing cycle is recommended to make certain no application functionality is impacted.

As with any application, mobile application testing is very important, as the clientele is frequently in the millions for a certain product — and a product with bugs is never appreciated. It often results in monetary losses, legal issues and irreparable brand image damage.

Basic differences between mobile and desktop application testing

A few obvious aspects that set mobile app testing apart from desktop testing:

  • On a desktop, the application is tested on a central processing unit. On a mobile device, the application is tested on handsets like Samsung, Nokia, Apple and HTC.
  • Mobile device screen size is smaller than desktop.
  • Mobile devices have less memory than a desktop.
  • Mobiles use network connections like 2G, 3G, 4G or WIFI whereas desktops use broadband or dial up connections.
  • The automation tool used for desktop application testing might not work on mobile applications.

Types of mobile app testing

To address all the above technical aspects, the following types of testing are performed on mobile applications:

  • Usability testing Used to ensure that the mobile app is easy to use and provides a satisfactory user experience to the customers.
  • Compatibility testing Testing of the application in different mobile devices, browsers, screen sizes and OS versions according to the requirements.
  • Interface testing Testing of menu options, buttons, bookmarks, history, settings, and navigation flow of the application.
  • Services testing Testing the services of the application online and offline.
  • Low-level resource testing Testing of memory usage, auto deletion of temporary files, local database growth issues, etc.
  • Performance testing Testing the performance of the application by changing the connection from 2G, 3G to WIFI, sharing the documents, battery consumption, etc.
  • Operational testing Testing of backup and recovery plan if battery goes down, or data loss while upgrading the application from the store.
  • Installation tests Validation of the application by installing/uninstalling it on the devices.
  • Security testing Testing an application to validate if the information system protects data or not.

Mobile application testing strategy

The test strategy should make sure that all the quality and performance guidelines are met. A few pointers in this area:

Œ #1 Selection of the devices

Analyze the market and choose those devices that are widely used. (This decision mostly relies on the clients. The client or app builders consider the popularity factor of each device, as well as the marketing needs for the application, to decide which handsets to use for testing.)

#2 Emulators

The use of these is extremely helpful in the initial stages of development, as they allow for quick and efficient checking of the app. An emulator is a system that runs software from one environment to another environment without changing the software itself. It duplicates the features and work of a real system.

Types of Mobile Emulators

Device emulator: provided by device manufacturers.

Browser emulator: simulates mobile browser environments.

Operating system emulator: Apple provides emulators for iPhones, as does Microsoft for Windows phones and Google for Android phones.

See the accompanying section for a sampling of freely available, online, easy-to-use mobile device emulators.

#3 Physical devices

After a satisfactory level of development is complete for the mobile app, you can move to test on physical devices for more real-life scenarios-based testing.

#4 Cloud computing-based testing

Cloud computing is something you should seriously consider. It is basically running devices on multiple systems or networks via the Internet, where applications can be tested, updated and managed. For testing purposes, it creates the web-based mobile environment on a simulator to access the mobile app.

Pros of Cloud Computing

  • Backup and recovery
    Cloud computing automatically takes a backup of your data from a remote location, making recovery and restoration of data easy. And, the storage capacity is unlimited.
  • Accessibility
    Clouds can be accessed from different devices and from anywhere.
  • Convenience
    Cloud computing is cost-efficient, easy to use, maintain and update.
  • Fast and quick deployment
  • Web-based interface
  • Efficiency
    Can run the same script on several devices in parallel.

Cons of Cloud Computing

  • Less control
    Since the application runs on a remote or third party environment, the user has limited control and access over the functions.
  • Internet connectivity issues
    The setup is on the Internet. Network issues affect the availability and functionality.
  • Security and privacy issues
    Cloud computing is Internet computing and nothing on the Internet is completely secure, so the risks of data hacking are higher.

#5 Automated vs. Manual testing

If the application contains new functionality, test it manually.

If the application requires testing once or twice, do it manually.

Automate the scripts for regression test cases. If regression tests are repeated, automated testing is perfect for that.

Automate the scripts for those complex scenarios which are time-consuming if executed manually.

Two kinds of automation tools are available to test mobile apps:

  • Object based mobile testing tools
    Automation by mapping elements on the device screen into objects. This approach is independent of screen size and mainly used for Android devices.
    Examples: ranorex, jamo solution
  • Image based mobile testing tools
    Create automation scripts based on the screen coordinates of elements.
    Examples: Sikuli, Egg Plant, RoutineBot


#6 Network configuration

This is also a necessary part of mobile testing. It’s important to validate the application on different networks like 2G, 3G, 4G or WIFI.

Test cases for testing a mobile app

In addition to functionality-based test cases, mobile application testing requires special test cases which should cover following scenarios:

  • Battery usage
    It’s important to keep a track of battery consumption while running applications on mobile devices.
  • Speed of the application
    The response times on different devices, with different memory parameters, with different network types, etc.
  • Data requirements
    For installation, as well as to verify that the user with a limited data plan will able to download and run it.
  • Memory requirements
    Again, to download, install and run.
  • Functionality of the application
    To ensure that the application is not crashing due to network failure or anything else.


Typical activities and proceedings in testing mobile applications

The scope of the testing depends on the amount of requirements to be checked or the extent of changes made to the app. If the changes are few, a round of sanity testing will do. In case of major and/or complex changes, a full regression is recommended.

An example application testing project

ILL (International Learn Lab) is an application designed to help the admin and publisher create websites in collaboration. Using a web browser, instructors choose from a set of features to create a class that meets their requirements.

Mobile Testing process:

Step #1 Identify the types of testing
As the ILL application is applicable to browsers, it’s mandatory to test this application on all supported browsers using different mobile devices. We need to do usability, functional and compatibility testing on different browsers with the combinations of manual and automated test cases.

Step #2 Manual and Automated testing
The methodology followed for this project is Agile, with an iteration of two weeks. Every two weeks, the dev team releases a new build to the testing team and the testing team runs their test cases on the QA environment. The automation team creates scripts for the set of basic functionality and runs the scripts that help determine if the new build is stable enough to test. The manual testing team tests the new functionality.

JIRA is used for writing of acceptance criteria, maintaining of test cases and logging/re-verification of defects. Once the iteration gets over, an iteration planning meeting is held where the dev team, product owner, business analyst, and QA team discuss what went well and what needs to improve.

Step #3 Beta Testing
Once the regression testing is completed by the QA team, the build moves into User Acceptance Testing (UAT). UAT is done by the client. They re-verify all the bugs to make sure every bug was fixed and the application is working as expected on every approved browser.

Step #4 Performance test
The performance testing team tests the performance of the web app using JMeter scripts and with different loads on the application.

Step #5 Browser testing
The web app gets tested across multiple browsers, using both different simulation tools as well as real physical mobile devices.

Step #6 Launch plan
After every 4th week the testing moves into staging, where a final round of end-to-end testing on these devices is performed to make sure the product is ready for production. And then, it goes Live!


Designing the right test strategy, choosing the right mobile simulators, devices and mobile testing tools can help ensure that we have 100% test coverage and help us include security, usability, performance, functionality and compatibility-based tests into our test suites.


This article originally appeared in Software Testing Help at http://www.softwaretestinghelp.com/beginners-guide-to-mobile-application-testing/


Nancy Ratnakar is a senior QA engineer in a multinational corporation. She has more than five years of experience in QA and mobile testing. With extensive experience in STLC, she is an expert at handling projects from the initial requirements stage to final release.

Nancy Ratnakar
is a senior QA engineer in a multinational corporation. She has more than five years of experience in QA and mobile testing. With extensive experience in STLC, she is an expert at handling projects from the initial requirements stage to final release.

The Related Post

I’ve spent the last six months or so testing mobile apps for both iOS and Android. Here’s eight of my key lessons learned: Automated UI testing tools for mobile apps are immature: whilst tools like WebDriver for automated UI testing of web apps are very mature, automated UI testing of native mobile apps is the ...
Testing appears to be the least popular topic in Android development circles based on the relatively few books on Android app testing. Most tend to focus on development because, unfortunately (but true), application testing isn’t be something most developers think much about, or if they do, they don’t do it systematically (I’m guilty of this ...
The outbreak of smartphones and tablets forces us to be digitally available with speed. Keeping pace with communication tool developments, Lindiwe Vinson defines the methods used at Organic, Inc. where she leads her team discovering bugs using various key programs for both PC and Mac platforms.
Organizations need to implement automated testing strategies designed specifically for mobile applications. Mobile device usage continues to gain momentum at increasing speed. Enterprises that delay mobile adoption face the danger of becoming competitively disadvantaged. But, before jumping in headlong, you need to be fully aware of the unique challenges that can arise when developing and implementing ...
Users aren’t likely to forgive and forget buggy apps. Mobile has big implications for business. The mobile experience is the customer experience, and you don’t get many second chances.
In the last issue on testing the SMAC stack we talked about the social and mobile aspects of testing. We will be referring to them in this article. In this issue part 2, we focus on the Analytics and Cloud aspect. The goal of this article is to understand a simple landscape of analytics and cloud.
This is the second part of a two part article that analyzes the impact of product development for the internet of things (IoT) on software testing.  Part one of this article (LogiGear Magazine, Sept 2014) gave a wide view on the IoT, embedded systems, and the device development aspects of testing on these projects. This ...
I am not a big fan of concepts which moves industry standards to IT. I am rather a Agile and Scrum guy. Managing multiple projects at once and trying to set a highest quality standard is a challenge and this book shows how industrial language can be translated into software development. I do not think that it ...
Manual testing teams may not be able to test all the processes with each build Test automation of applications has been around for many years. There are many of us in the automated testing field that started very early in the test automation phase, but the introduction of mobile devices has brought on a new angle ...
Mobile is no longer an area that a few UX people specialize in, and we need to start designing and testing everything for smartphones and tablets as well as computers. If you’re new to mobile usability testing, fear not. It is not as hard as you might think but there are some key differences from ...
LogiGear Magazine, September 2015: Mobile Testing
Steps that will enable you to identify the weaknesses of your new app, its vulnerabilities and strengths. So you’ve just finished developing a nifty, customisable app that can help farmers track their produce from source to market via their mobile phone. You’re elated and want to get started marketing it right away. Not to burst ...

Stay in the loop with the lastest
software testing news