Letter from the Editor – September 2015

In every year since 2011, we have devoted one edition of our magazine to the topic of mobile testing. In this year’s issue on mobile, we focus on testing from the point of view of the user experience.

Most teams start with UI testing, and it may seem basic — until you look at the importance and uniqueness of UI testing for mobile apps and websites. There’s a lot to understand, but the most important thing is that mobile users aren’t very bug tolerant. Their expectation is flawless performance.

First, the problem of mobile responsiveness. The Wikipedia definition of responsive web design (RWD) is: …”an approach to web design aimed at crafting sites to provide an optimal viewing and interaction experience — easy reading and navigation with a minimum of resizing, panning, and scrolling — across a wide range of devices (from desktop computer monitors to mobile phones).”

Responsiveness testing is key for any mobile software product. That is the ability for your site UI to be accessed by all the defined devices with varying screen sizes and orientations (portrait or landscape). This is different from functionality or even UI testing, since UI testing focuses on the correct behavior of UI controls such as combo boxes, text boxes, buttons, or links. Responsiveness testing examines the layout and look and feel of, for example, a 12.9” iPad vs. a 9.7” iPad, not UI functionality.

Second, what mobile devices do you test? I just had a new client request that I specify what platforms to use for mobile testing. That is a huge question with no easy answer. The mobile world is increasingly fractured. The devices, operating systems and browsers, rather than converging, are becoming increasingly diversified. There are so many manufacturers making devices with varying hardware components. For example, if your app uses geolocation, do you have to test devices’ gyroscopes or accelerometers, gravity sensors, compass sensors and magnetometers? It’s a complicated answer.

So many devices, and so little time. How many devices do you need to test? How and where will you execute your tests? On real devices? Buy a Cloud service? Use emulators? At the UI? At an API? We have been having these problems in mobile testing since it began. The same issues are here today and it is getting ever more complex as the device world diversifies.

These problems would diminish, or go away entirely, with a convergent mobile device market. But not only is the device/hardware market fracturing, the OS world is getting more complicated as well. For the Android OS alone, the various versions of Lollipop, KitKat and Jellybean all have significant usage, with lesser use of Ice Cream Sandwich and Gingerbread. They are definitely not all the same. So for Android, what do you test?

Companies continue to struggle with a mobile testing strategy. There are more questions than answers, and this is not necessarily a bad thing!

As I learned from my first and greatest testing manager, Cem Kamer, “half of our job is education.” Wow.

Bring your test strategy and coverage plans to the team. Educate your team to these complexities. Marketing, POs (product owners), product management, as well as programmers have to be involved in your decision-making. What you test — the hardware, sensors and screen size, and OS combinations — as well as how to test — emulators, cloud, real devices — needs to be knowledgeably and openly discussed and then agreed upon.

In these Agile and Lean days of no or few test plans, someone has to be having these conversations; someone has to be asking these questions. That person is you.

Our last issue of 2015 will be on Test Automation. That issue will also contain the Editorial
Calendar for 2016. Until then, good luck and have fun testing!

Michael Hackett
Michael is a co-founder of LogiGear Corporation, and has over two decades of experience in software engineering in banking, securities, healthcare and consumer electronics. Michael is a Certified Scrum Master and has co-authored two books on software testing. Testing Applications on the Web: Test Planning for Mobile and Internet-Based Systems (Wiley, 2nd ed. 2003), and Global Software Test Automation (Happy About Publishing, 2006). He is a founding member of the Board of Advisors at the University of California Berkeley Extension and has taught for the Certificate in Software Quality Engineering and Management at the University of California Santa Cruz Extension. As a member of IEEE, his training courses have brought Silicon Valley testing expertise to over 16 countries. Michael holds a Bachelor of Science in Engineering from Carnegie Mellon University.

The Related Post

I remember the times when test teams sat in their own area and we were not allowed to “bother” developers.
Everything is mobile. What else can we say? Everything. If your product or service is currently not, it will be very soon. As Apple says: “There’s an app for that.” There is an app for everything. The race for mobile apps has consumed the software development world. I did a few projects at Palm Computing in the ...
Methods and strategy have been my favorite topics since I started working in testing. It’s essentially engineering problem-solving. It’s both looking for efficiency and attempting to measure effectiveness. So, how do we develop a set of practices to solve our Software Testing engineering problems?
DevOps can be a big scary thing. Culture change, constant collaboration— whatever that means— a big new set of tools… it’s a lot. What most teams want is to have a smooth running software development pipeline. I have stopped using the phrase “DevOps,” and now I say “Continuous Delivery.” There are many reasons for this.
I led the Editor’s Note in our very first mobile issue with “Everything is mobile”, but it is now way beyond what we thought. Mobile has come to mean only the smart phone, mobility is the word that describes everything a smart phone enables you to do. Mobility is more than a device! Mobility is ...
Test automation is a big topic. There are so many different areas to talk about: tool choice, jumpstart, cross platform, services, cloud… Each of these areas have changed so much in the recent past that they could each be worth their own magazine issue.
Every organization goes through times when the internal, or home team, cannot execute the testing project easily or quickly enough. The reasons are many, from the lack of an effective test strategy to low automation engineering skill, to staff positions going unfilled due to a great job market. With everyone working and very few people ...
On the whole, everyone wants to do a great job, have a better work environment, happy clients and customers, and to be employed by a company earning lots of money. All great goals! But this is not always the case. When it is not, you can suggest process improvements, better tool use, different estimating techniques, ...
There has been a tectonic shift in software development tools in just the past few years. Agile practices and increasingly distributed teams have been significant factors but, in my opinion, the main reason is a new and more intense focus on tools for testing driven by more complex software and shorter development cycles. There have ...
Continuous Testing… what is it? When we first decided to do a magazine issue dedicated to the DevOps practice of Continuous Testing, I joked with someone: “It’s about testing continuously.” And their reply was: “Yeah. What else would it be?” I was joking, but clearly the joke didn’t land. Continuous Testing is about testing continuously, ...
We launched the first ever software testing conference in Vietnam, VISTACON. It was a resounding success, with well over 200 participants and 20+ speakers from around the globe; each speaking on a wide range of cutting-edge testing topics. In this month’s magazine, we have uploaded several video recordings of event presentations – giving our readers ...
This is our third issue concerning topics of Continuous Delivery (CD) and DevOps with the inclusion of Continuous Testing. DevOps has been around for a while and I hope the period of buzz is over and companies moving towards building a development pipeline have begun their process, including changing their test strategies.

Leave a Reply

Your email address will not be published. Required fields are marked *

Stay in the loop with the lastest
software testing news