What Matters in Mobile Testing?

Devices matter. We don’t yet trust the mobile devices like we trust desktops and laptops. In the course of testing traditional web applications, rarely do you have to think about the model of the actual machine. In mobile, however, the behavior of an application can vary from device to device. You can no longer just think about browser/OS combinations.

Mobile emulators aren’t good enough. They get you part way there, but you still have to physically test on many devices.

The version of the OS can be quite important. Just as you think about the physical device, you also must consider how nuances of each OS version can affect application behavior: iOS 3, 4 or 5; Android 2.1, 2.2 , or 2.3; Blackberry 5, 6, or 7; Windows Mobile 7.0 or 7.5; etc. Add that to your test matrix.

Mobile web apps can become native apps. Tools like PhoneGap and Appcelerator enable web applications to be compiled into native applications. Now you might need to test a version of the application that runs in browsers and a version that runs natively.

Native phone features must now be considered. With the ability to turn web applications into native applications, this opens up access to native phone features, like the camera, accelerometer, address book, etc. These are not things we are all that familiar with.

Mobile web applications tend to be heavy on the JavaScript. Many factors lead to a thick-client architectural style for mobile applications. Business logic is pushed to the client-side—i.e. JavaScript —and the server-side becomes a set of services. Suffice to say that you now may find yourself much more concerned about how to test JavaScript.

Offline capability is important. This may become a first class concern. There are several reasons why a phone can lose connectivity, such as Airplane mode and driving into a tunnel, and mobile applications must be designed to deal with this fact—yet, another thing to add to your testing list.

Connection quality is unpredictable. Remember when you had to worry about dial-up users? It’s like that. Even if you assume your target user base will have at least 3G, connectivity can degrade or be lost at any time. One minute you’re on Wi-Fi, the next on 3G, and the next on Edge.

Data plans might be metered. Many users do not have unlimited data usage. Now the application architecture and design has to take into account how it will need to respect the user’s data plan. And you have to test it.

Instrumentation and test tool support is lagging on mobile devices. Any time there is emerging technology, the development of supporting tools (like testing tools) lags. For example, there are not currently a great many mature tool options available to automate tests of mobile applications across the spectrum of devices, OSes, browsers, etc.

The network provider might matter. Perhaps a particular network provider throttles connection speeds. Perhaps a particular network provider configures its phones in a particular way that might be important to know.

You have a lot to consider building your test matrix for testing mobile applications. The test strategies are still evolving. We are all still figuring out what matters in mobile application testing. Good luck!

John Roets
John Roets has been providing software engineering leadership, specifically to application development for the web, since 1998. John is currently a Senior Architect at ITX Corporation in Rochester, NY; providing solutions to a multitude of clients, focusing his attention (at the moment) on mobile applications. He is just as likely to engage you in discussion about design patterns, databases, and the latest web technologies as he is about business strategy, software development methodologies and organizational structure.  John can be contacted through http://jroets.blogspot.com/</em
John Roets
John Roets has been providing software engineering leadership, specifically to application development for the web, since 1998. John is currently a Senior Architect at ITX Corporation in Rochester, NY; providing solutions to a multitude of clients, focusing his attention (at the moment) on mobile applications. He is just as likely to engage you in discussion about design patterns, databases, and the latest web technologies as he is about business strategy, software development methodologies and organizational structure.

The Related Post

The most significant facet of mobile testing is understanding the mobile ecosystem. Mobile applications are growing in use and sophistication along with the speed of the networks and the increasing power of the devices. To be sure, mobile application testing can be done with minimal automation, but a high level of test automation will help ...
LogiGear Magazine – September 2013 – Mobile Testing
This article will cover 10 common mobile app testing mistakes to avoid when you are a software tester working in a mobile app testing and development environment. The 10 points may help you to start your mobile testing activities if you are new to mobile testing or they may help you to recap your existing mobile testing ...
Mobile is no longer an area that a few UX people specialize in, and we need to start designing and testing everything for smartphones and tablets as well as computers. If you’re new to mobile usability testing, fear not. It is not as hard as you might think but there are some key differences from ...
The outbreak of smartphones and tablets forces us to be digitally available with speed. Keeping pace with communication tool developments, Lindiwe Vinson defines the methods used at Organic, Inc. where she leads her team discovering bugs using various key programs for both PC and Mac platforms.
Users aren’t likely to forgive and forget buggy apps. Mobile has big implications for business. The mobile experience is the customer experience, and you don’t get many second chances.
                                                                                                              ...
CEO and founder of mVerify Corporation, Robert V. Binder tackles questions from field testers regarding such issues as strategic considerations when dealing with single stack apps versus globalized enterprise mobile apps, and methods and tools that developers and testers should be aware of. He also offers his own advice from lessons learned from experience. 1. ...
To ensure the success of an app, QA must be involved in all stages of development. Quality Assurance (QA) plays a vital role in the development of mobile applications, but many overlook the critical nature of this piece of the app development process. To ensure the success of an app, QA must be involved in ...
Strategies to Approach Mobile Web App Testing Mobile web technology has been continuously changing over the past few years, making “keeping up” challenging. In this article, Raj Subramanian covers the latest trends and changes happening in the mobile web and how testers can prepare for them.
Test engineers face a rapidly changing mobile application landscape, making mobile test automation a necessity. We know that mobile apps are becoming increasingly complex along with the technological advances in tablets and smartphones. Test engineers have to address multiple challenges while testing data-centric mobile apps, including the growing diversity of device features, platforms, and technologies. Fortunately, ...
 LogiGear_Magazine_October_2014_Testing_Smart_and_Mobile

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe