What Matters in Mobile Testing?

Devices matter. We don’t yet trust the mobile devices like we trust desktops and laptops. In the course of testing traditional web applications, rarely do you have to think about the model of the actual machine. In mobile, however, the behavior of an application can vary from device to device. You can no longer just think about browser/OS combinations.

Mobile emulators aren’t good enough. They get you part way there, but you still have to physically test on many devices.

The version of the OS can be quite important. Just as you think about the physical device, you also must consider how nuances of each OS version can affect application behavior: iOS 3, 4 or 5; Android 2.1, 2.2 , or 2.3; Blackberry 5, 6, or 7; Windows Mobile 7.0 or 7.5; etc. Add that to your test matrix.

Mobile web apps can become native apps. Tools like PhoneGap and Appcelerator enable web applications to be compiled into native applications. Now you might need to test a version of the application that runs in browsers and a version that runs natively.

Native phone features must now be considered. With the ability to turn web applications into native applications, this opens up access to native phone features, like the camera, accelerometer, address book, etc. These are not things we are all that familiar with.

Mobile web applications tend to be heavy on the JavaScript. Many factors lead to a thick-client architectural style for mobile applications. Business logic is pushed to the client-side—i.e. JavaScript —and the server-side becomes a set of services. Suffice to say that you now may find yourself much more concerned about how to test JavaScript.

Offline capability is important. This may become a first class concern. There are several reasons why a phone can lose connectivity, such as Airplane mode and driving into a tunnel, and mobile applications must be designed to deal with this fact—yet, another thing to add to your testing list.

Connection quality is unpredictable. Remember when you had to worry about dial-up users? It’s like that. Even if you assume your target user base will have at least 3G, connectivity can degrade or be lost at any time. One minute you’re on Wi-Fi, the next on 3G, and the next on Edge.

Data plans might be metered. Many users do not have unlimited data usage. Now the application architecture and design has to take into account how it will need to respect the user’s data plan. And you have to test it.

Instrumentation and test tool support is lagging on mobile devices. Any time there is emerging technology, the development of supporting tools (like testing tools) lags. For example, there are not currently a great many mature tool options available to automate tests of mobile applications across the spectrum of devices, OSes, browsers, etc.

The network provider might matter. Perhaps a particular network provider throttles connection speeds. Perhaps a particular network provider configures its phones in a particular way that might be important to know.

You have a lot to consider building your test matrix for testing mobile applications. The test strategies are still evolving. We are all still figuring out what matters in mobile application testing. Good luck!

John Roets
John Roets has been providing software engineering leadership, specifically to application development for the web, since 1998. John is currently a Senior Architect at ITX Corporation in Rochester, NY; providing solutions to a multitude of clients, focusing his attention (at the moment) on mobile applications. He is just as likely to engage you in discussion about design patterns, databases, and the latest web technologies as he is about business strategy, software development methodologies and organizational structure.  John can be contacted through http://jroets.blogspot.com/</em
John Roets
John Roets has been providing software engineering leadership, specifically to application development for the web, since 1998. John is currently a Senior Architect at ITX Corporation in Rochester, NY; providing solutions to a multitude of clients, focusing his attention (at the moment) on mobile applications. He is just as likely to engage you in discussion about design patterns, databases, and the latest web technologies as he is about business strategy, software development methodologies and organizational structure.

The Related Post

25% of Americans own a tablet. Up from 11% of U.S. adults in July of 2011 to 18% in January of 2012. – Pew Internet & American Life Project Nigeria has close to 100 million mobile phone lines, making it Africa’s largest telecoms market. – Nigerian Communications Commission Google plans to sell 200 million Android ...
Strategies to Approach Mobile Web App Testing Mobile web technology has been continuously changing over the past few years, making “keeping up” challenging. In this article, Raj Subramanian covers the latest trends and changes happening in the mobile web and how testers can prepare for them.
I’ve spent the last six months or so testing mobile apps for both iOS and Android. Here’s eight of my key lessons learned: Automated UI testing tools for mobile apps are immature: whilst tools like WebDriver for automated UI testing of web apps are very mature, automated UI testing of native mobile apps is the ...
A fully integrated test automation platform for mobile testing Mobile application testing is taking on greater significance as just about everything done on the desktop is transitioning to mobile. It’s taken a while for mobile testing in the consumer space to be made much of a priority. Now that sensitive data is being passed back ...
Removing the barriers to move and better mobile testing. Over the last decade, application testing has continually proved itself to be an important concern. When done well, testing can drastically reduce the number of bugs that make it into your release code (and thus actually affect your users). In addition, good testing approaches will help your ...
In today’s mobile-first world, a good app is important, meaning an effective Mobile Testing strategy is  essential.  
What you need to know for testing in the new paradigm This two part article analyzes the impact of the Internet of Things (IoT) product development on traditional testing. Part one of this series starts with a wide view on the IoT, embedded systems and device development aspects of testing. Part two, to be published ...
Don’t make the mistake of assuming too many similarities. It is common knowledge that mobile applications don’t function in the same way as their web-based counterparts. The user experience is affected by a few other factors such as device and network capability. If you are building out a performance testing strategy for your mobile website ...
Gal Tunik explains the necessity of test automation in mobile applications, citing the unique challenges that that platform raises, including the intricate differences between mobile and desktop interfaces, and performance test results for applications as well as mobile networks. It is no surprise in this day and age of instant-on gratification that mobile devices are ...
LogiGear Magazine – September 2013 – Mobile Testing
Mobile is no longer an area that a few UX people specialize in, and we need to start designing and testing everything for smartphones and tablets as well as computers. If you’re new to mobile usability testing, fear not. It is not as hard as you might think but there are some key differences from ...
LogiGear_Magazine_December 2016_Riding the New Software Testing Wave  

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe