What Matters in Mobile Testing?

Devices matter. We don’t yet trust the mobile devices like we trust desktops and laptops. In the course of testing traditional web applications, rarely do you have to think about the model of the actual machine. In mobile, however, the behavior of an application can vary from device to device. You can no longer just think about browser/OS combinations.

Mobile emulators aren’t good enough. They get you part way there, but you still have to physically test on many devices.

The version of the OS can be quite important. Just as you think about the physical device, you also must consider how nuances of each OS version can affect application behavior: iOS 3, 4 or 5; Android 2.1, 2.2 , or 2.3; Blackberry 5, 6, or 7; Windows Mobile 7.0 or 7.5; etc. Add that to your test matrix.

Mobile web apps can become native apps. Tools like PhoneGap and Appcelerator enable web applications to be compiled into native applications. Now you might need to test a version of the application that runs in browsers and a version that runs natively.

Native phone features must now be considered. With the ability to turn web applications into native applications, this opens up access to native phone features, like the camera, accelerometer, address book, etc. These are not things we are all that familiar with.

Mobile web applications tend to be heavy on the JavaScript. Many factors lead to a thick-client architectural style for mobile applications. Business logic is pushed to the client-side—i.e. JavaScript —and the server-side becomes a set of services. Suffice to say that you now may find yourself much more concerned about how to test JavaScript.

Offline capability is important. This may become a first class concern. There are several reasons why a phone can lose connectivity, such as Airplane mode and driving into a tunnel, and mobile applications must be designed to deal with this fact—yet, another thing to add to your testing list.

Connection quality is unpredictable. Remember when you had to worry about dial-up users? It’s like that. Even if you assume your target user base will have at least 3G, connectivity can degrade or be lost at any time. One minute you’re on Wi-Fi, the next on 3G, and the next on Edge.

Data plans might be metered. Many users do not have unlimited data usage. Now the application architecture and design has to take into account how it will need to respect the user’s data plan. And you have to test it.

Instrumentation and test tool support is lagging on mobile devices. Any time there is emerging technology, the development of supporting tools (like testing tools) lags. For example, there are not currently a great many mature tool options available to automate tests of mobile applications across the spectrum of devices, OSes, browsers, etc.

The network provider might matter. Perhaps a particular network provider throttles connection speeds. Perhaps a particular network provider configures its phones in a particular way that might be important to know.

You have a lot to consider building your test matrix for testing mobile applications. The test strategies are still evolving. We are all still figuring out what matters in mobile application testing. Good luck!

John Roets
John Roets has been providing software engineering leadership, specifically to application development for the web, since 1998. John is currently a Senior Architect at ITX Corporation in Rochester, NY; providing solutions to a multitude of clients, focusing his attention (at the moment) on mobile applications. He is just as likely to engage you in discussion about design patterns, databases, and the latest web technologies as he is about business strategy, software development methodologies and organizational structure.  John can be contacted through http://jroets.blogspot.com/</em
John Roets
John Roets has been providing software engineering leadership, specifically to application development for the web, since 1998. John is currently a Senior Architect at ITX Corporation in Rochester, NY; providing solutions to a multitude of clients, focusing his attention (at the moment) on mobile applications. He is just as likely to engage you in discussion about design patterns, databases, and the latest web technologies as he is about business strategy, software development methodologies and organizational structure.

The Related Post

  Mobile analytics experts Julian Harty and Antoine Aymer have teamed up to deliver a 161-page handbook designed to help you “enhance the quality, velocity, and efficiency of your mobile apps by integrating mobile analytics and mobile testing”.
CEO and founder of mVerify Corporation, Robert V. Binder tackles questions from field testers regarding such issues as strategic considerations when dealing with single stack apps versus globalized enterprise mobile apps, and methods and tools that developers and testers should be aware of. He also offers his own advice from lessons learned from experience. 1. ...
Users aren’t likely to forgive and forget buggy apps. Mobile has big implications for business. The mobile experience is the customer experience, and you don’t get many second chances.
Mobile is no longer an area that a few UX people specialize in, and we need to start designing and testing everything for smartphones and tablets as well as computers. If you’re new to mobile usability testing, fear not. It is not as hard as you might think but there are some key differences from ...
In today’s mobile-first world, a good app is important, meaning an effective Mobile Testing strategy is  essential.  
I am not a big fan of concepts which moves industry standards to IT. I am rather a Agile and Scrum guy. Managing multiple projects at once and trying to set a highest quality standard is a challenge and this book shows how industrial language can be translated into software development. I do not think that it ...
LogiGear_Magazine_December 2016_Riding the New Software Testing Wave  
Here’s some good news: Jonathan Kohl has a new book out titled Tap Into Mobile Application Testing. This is a Leanpub release and is still in Beta, but the beauty of buying through Leanpub – you get all the author updates as they make them! However I will be keeping an eye out for the print-on-demand ...
Manual testing teams may not be able to test all the processes with each build Test automation of applications has been around for many years. There are many of us in the automated testing field that started very early in the test automation phase, but the introduction of mobile devices has brought on a new angle ...
What you need to know for testing in the new paradigm This two part article analyzes the impact of the Internet of Things (IoT) product development on traditional testing. Part one of this series starts with a wide view on the IoT, embedded systems and device development aspects of testing. Part two, to be published ...
The mobile application ecosystem is very dynamic. OEMs are launching new devices and new customization, and new OS versions are delivered every now and then. This is the constant challenge that most enterprises face.
To start with, we need a Test schedule. The same is created in the process of developing the Test plan. In this schedule, we have to estimate the time required for testing of the entire Data Warehouse system. There are different methodologies available to create a Test schedule. None of them are perfect because the ...

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe