Extending Test Automation to Mobile Applications

Gal Tunik explains the necessity of test automation in mobile applications, citing the unique challenges that that platform raises, including the intricate differences between mobile and desktop interfaces, and performance test results for applications as well as mobile networks.

It is no surprise in this day and age of instant-on gratification that mobile devices are gaining momentum. People have become accustomed to accessing information when they want it, wherever they are, and the mobile device is one way to stay connected. Across all industries, companies are realizing that to remain competitive, they must provide customers with ways to make everyday connectivity to the world easier. However, in order to create a successful mobile application, it is important that companies conduct the necessary tests to ensure that it is market-ready.

Accelerating this momentum is the hype surrounding such new technologies as the superfast 4G network, bar codes, Near Field Communication (NFC) and augmented reality. These all provide companies with attractive business opportunities and spur innovation. More and more, companies are increasing efforts and adding resources to develop mobile applications. Twenty-four percent of executives in charge of their companies’ mobile strategies reported that they would more than double their spending in the upcoming year, according to a recent independent Forrester Research survey1 of mobile executives. Additionally, 46 percent stated they would increase their budgets.

Mobile applications have revolutionized how people conduct business. For example, using a smartphone, a retail store manager can check the exact time product inventory is due in her store. Nurses can quickly access patients’ health records from the examining room and sales representatives are able to multi-task and report on deal statuses while on their way to the airport.

However, as with any emerging technology, a number of unique challenges can arise when developing and implementing mobile applications. Mobile applications don’t simply imitate the desktop environment—they have their own user interface (UI) requirements, business process flows, and infrastructure dependencies.

In addition, there are a growing number of mobile devices— smartphones, PDAs, tablets, and specialized devices such as a delivery clerk’s scanning and tracking handheld. In recent years, there have also been a number of different mobile operating systems. To add even more complexity, there is a choice of implementation options—such as thick vs. thin client. All of this increases the strain on IT to build, port and

maintain mobile applications. It also heightens the risk of quality and performance problems.

 

Functional Validation

One way to curtail glitches is to make certain mobile applications go through functional validation. This requires testing applications for mobile devices, similar to how desktop-based applications are tested, to ensure usability. Functional validation is based on the concepts of collaboration, test asset management, reusability, accurate reporting and analysis. Similar to the desktop application world, the bulk of testing on mobile devices is currently done manually.

There are some who are skeptical about the accuracy of manual testing. Can a test engineer simply drive to a remote location and log into the credit union’s website to check his account balance? Or, can he see whether he still gets a good network connection while delivering a package to a far-away golf course? The problem with manually testing mobile applications is the lack of coverage and scalability, as well as the associated costs, especially if an application needs to be re-tested multiple times to ensure development and quality. While a test engineer can manually key in a handful of transactions, he cannot continuously test and monitor the connection for application availability or test for all possible scenario permutations. Additionally, a tester is not able to quickly identify problems, fix them, and re-run all the tests required to validate the fix.

Another popular debate is whether testing should be done on an application’s physical device versus using emulators. However, the question should really be: how should we test both native applications and mobile web applications to achieve the required quality?

Many industry experts argue that using handsets for testing, as opposed to testing the application on a normal browser or emulator, requires an overhead of logistics. However, in order to ensure that they function and perform properly, it is imperative that applications be tested on mobile handsets. There are several reasons for this, ranging from the theoretical to the practical:

· For one, usability testing on emulators and browsers with any extensions do not represent what will be seen on the actual device. With Android specifically, emulated “vanilla” environments almost never represent the actual behavior of the phone, as vendors tend to modify the “source code” of the handset and add their own customization layers.

  • Environment-related testing is critical as mobile applications rely on the location, accelerometer and network. Simply put, the test cannot be done in a simulated environment.
  • Another factor is interoperability. A mobile handset “lives” in a communication environment and behaves accordingly. For example, an incoming phone call always receives priority over any application that is in use. Testing scenarios that see what happens to an application when a phone call is received is critical, as this is an everyday occurrence.
  • As many mobile users travel internationally, the proper network-related testing must be conducted. For instance, an application developed for Vodafone in the UK does not necessarily work on the Verizon network in the US. Network neutrality has not been integrated in the cellular world, which means traffic manipulation and modification is still very common in the cellular environment.
  • A common concern amongst mobile users is security. People are sensitive to data such as bank account numbers that remain on handsets, or passwords that are displayed on the screen. Testing for these types of security concerns in a simulated environment is not a good use of time, because what is required to be tested is the behavior of the actual handset. Adding to the challenge, not all handsets have the same security designs, so each device must be individually tested.

 

Performance Validation

A second aspect of the proliferation of mobile applications is the need for performance validation. Applications must be validated to ensure performance expectations are met. The user expectations from these applications may even be higher than those of their desktop counter-parts. Performance validation for mobile applications is crucial and, at times, more complex than that of desktop applications.

Mobile applications, by their very nature, answer a desire as well as a need for access to information “24/7”. Quite often they are built to provide access to a critical subset of desktop applications. These micro-business processes must always be readily available, to ensure businesses continue to function even when no one is in the office.

Another important performance factor is that mobile applications are affected by mobile network conditions, such as bandwidth limitations, which are more pronounced than those of land-line networks. These restrictions may adversely affect the communication between a handset and the back-end servers. Automating performance validation is needed to reach the large number of simulated users that will be accessing the tested application at any given moment. This can be achieved through simulated users running on emulators and browsers. Without this kind of automation, it is virtually impossible to create a realistic load on the application servers and infrastructure.

When replicating the appropriate network conditions, it must be ensured that a simulated mobile user is experiencing the same bandwidth throttling and network delays that a real mobile user would. In order to complete the test, real handsets executing the relevant business processes should be running parallel to the automated performance test. This allows the end user experience to be measured, while also testing the behavior of the application’s servers and infrastructure under load.

To appropriately test mobile applications, it is best to bring together a large number of browsers and emulator-based virtual users, simulations of mobile network conditions, as well as actual handsets to test business processes. This not only ensures performance validation, but provides comprehensive and accurate end-to-end results for mobile applications.

Conclusion

Automated functional and performance testing has a proven track-record in desktop-based applications. It helps companies deploy higher-quality software applications, reduce business risk, and accelerate problem-to-resolution. This ultimately helps organizations avoid costly interruptions and errors. The growing demand for instant information, especially through mobile applications, should encourage organizations to adopt automated testing strategies and solutions that are designed specifically for their mobile application needs.

1. Forrester Research, Inc. “2011 Mobile Trends,” January 24, 2011

 

Gal Tunik
Gal Tunik explains the necessity of test automation in mobile applications, citing the unique challenges that that platform raises, including the intricate differences between mobile and desktop interfaces, and performance test results for applications as well as mobile networks.
Edward Hill
Edward has worked at HP since 2007 is currently Sector Head—Retail and Consumer Goods at HP Software. His responsibilities include: the vertical transformation of the UK go-to-market strategy, sales execution for HP Software and to build a trusted, secure and vertically credible external perception of HPSW.

The Related Post

What you need to know to get started Introduction to mobile application testing Gone are the days when the telephone was an appliance that sat in a corner and had to ring to get our attention, or a computer was a machine only few people used — these devices are now an extension of our ...
What you need to know in order to have effective and reliable Test Automation for your mobile apps I realized that Test Automation interfaces are pivotal to effective and efficient Test Automation, yet very few people trying to test their mobile apps seemed to know how their automated tests connected with the apps they wanted ...
Users aren’t likely to forgive and forget buggy apps. Mobile has big implications for business. The mobile experience is the customer experience, and you don’t get many second chances.
Organizations need to implement automated testing strategies designed specifically for mobile applications. Mobile device usage continues to gain momentum at increasing speed. Enterprises that delay mobile adoption face the danger of becoming competitively disadvantaged. But, before jumping in headlong, you need to be fully aware of the unique challenges that can arise when developing and implementing ...
Great mobile testing requires creativity to find problems that matter. I’d like to take you through the thought process of testers and discuss the types of things they consider when testing a mobile app. The intention here is to highlight their thought processes and to show the coverage and depth that testers often go to.
Drawing from the Greek mythology of the lotus eaters, Anne-Marie Charrett warns testers to be weary of enjoying early success too soon upon finding high impact bugs.
In the last issue on testing the SMAC stack we talked about the social and mobile aspects of testing. We will be referring to them in this article. In this issue part 2, we focus on the Analytics and Cloud aspect. The goal of this article is to understand a simple landscape of analytics and cloud.
Removing the barriers to move and better mobile testing. Over the last decade, application testing has continually proved itself to be an important concern. When done well, testing can drastically reduce the number of bugs that make it into your release code (and thus actually affect your users). In addition, good testing approaches will help your ...
                                                                                                              ...
To help testers gain an edge, here’s a list of free resources Mobile testing is making leaps and bounds of progress in the overall testing space. As this field is highly dynamic, a tester must constantly evolve and improvise his or her knowledge of mobile testing. To help software testers gain an edge, I have compiled the following list ...
  Mobile analytics experts Julian Harty and Antoine Aymer have teamed up to deliver a 161-page handbook designed to help you “enhance the quality, velocity, and efficiency of your mobile apps by integrating mobile analytics and mobile testing”.
Here’s some good news: Jonathan Kohl has a new book out titled Tap Into Mobile Application Testing. This is a Leanpub release and is still in Beta, but the beauty of buying through Leanpub – you get all the author updates as they make them! However I will be keeping an eye out for the print-on-demand ...

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news

Subscribe