Mobile Usability Testing: Traditional Methods No Longer Hold Water

James Hoshor | August 28, 2017 | Mobile App Development Mobile Strategy

Strange as it may sound, mobile testing is the one key area where today’s mobile enterprises continue to under-perform. This is primarily the fault of the testers themselves, who are forced to conduct the bare minimum or too-rudimentary levels of testing. Most often this is due to a lack of testing resources, lack of experience and/or the absence of a solid mobile testing strategy and plan.

The focus of this blog is not to solve all the challenges mobile testing can present (that would require an entire book), but rather to focus on the mobile testing methods and tools testing organizations use today and to describe why these practices are insufficient and explain why it is critical for companies to pursue a more rigorous testing strategy.

One of the main testing methods I will focus on is usability testing. Often overlooked, this is actually the most important mobile testing area organizations need to pay much greater attention to.

Most testing organizations utilize the standard usability testing methods: focus groups and app monitoring, along with traditional quantitative analytics. While these may have formerly reflected testing best practices, today’s mobile behaviors and expectations have radically changed, a result of the continuous evolution of mobile technology and the myriad new ways users can now interact with mobile. Because of this continuous evolution in technology and user behavior, testing organizations must also evolve their mobile testing solutions to ensure they continue to deliver the most intuitive, up-to-date experience possible.

So which traditional usability testing methods are no longer sufficient? First off, the golden rule of testing still holds true: real people on real devices is still the best way to test. However, our experience has revealed a problem. When you ask someone to test a mobile app, that person won’t use the app the same way typical users would who just downloaded and started using the app, hoping to enjoy the newly mobilized and streamlined business processes they were already familiar with. This is what we keep seeing, time and time again. When assigned the task of testing an app, the tester’s mindset and behavior changes, deviating significantly from that of the average user. This includes the length of time spent on each screen, the process flow steps they follow, even the gestures they use to navigate the app. So from a technical perspective, it’s imperative that you realize your mobile app will likely yield different results in a controlled test than when it’s being used under normal user conditions in the typical work environment.

Another problem: a recent Localytics survey found 23% of mobile users stop using an app after the first use. In other words, if your app isn’t intuitive and easy to use, mobile users simply won’t use it. The average mobile user has 30 installed applications but uses fewer than 5 regularly. These findings help illustrate why usability testing (and of course other mobile testing methods) is critical.

So what’s today’s mobile enterprise to do? Well, one way organizations are going above and beyond with their usability testing methods is in the use of new qualitative analytics tools that track and produce touch-heat maps. These tools aggregate all gesture data (e.g. taps, swipes, pinches, etc.) used to interact with the app. User interactions are captured and presented visually as a transparent heat map layer placed atop the mobile app itself. This enables testers, etc. to visualize exactly where and how users have interacted with the mobile app. Frequency of interaction is depicted using a gradient color scheme—blue depicts the least frequent interactions and red represents the highest area of interactions. Below are examples of this solution from Appsee and HeatData.

mobile usability testingmobile usability testing
Image Source: Appsee                  Image Source: HeatData

The main goal of any mobile solution should be to make it intuitive and easy to use. That said, one key area organizations need to pay more attention to when performing usability testing is the unresponsive gesture: the moment a mobile user interacts with your app but their gesture receives no response. Many things could cause this. Either the screen is greasy, the app has a bug, or the user is trying to swipe when a tap is required. The point is unresponsive gestures should not be ignored since they represent another source of user frustration and another reason users might delete your mobile app (and post poor reviews in app stores or via social media), never to be installed again.

Another reason to move beyond typical usability testing methods and to incorporate a tool like touch-heat maps to track mobile interactions is the many different form factors mobile devices use today (e.g. watches, phones, tablets, etc.). Leveraging these new usability testing methods will help developers identify if mobile app elements appear off-screen or if different form factors result in poor UI experiences. As a mobile strategist I see more and more companies implementing mobile application development platform (MADP) solutions that enable them to deploy a single code base to multiple platforms and devices (native, hybrid and web). While this is becoming more of the norm in mobile app development (and is an arguably sound strategy), deploying to different platforms and devices creates additional complexities when it comes to mobile testing, especially usability testing.

After a decade of mobile app development advancements and the continuous evolution of mobile technology, the one thing we know for sure is that mobile user expectations and behaviors will also continue to evolve.

To ensure your mobile users have great experiences with your mobile solutions, you need a well-defined mobile testing strategy that is reviewed regularly and incorporates testing methods and tools that enable you to go above and beyond to make sure your app isn’t reviewed poorly and deleted by users. Propelics can help you develop your ideal mobile testing strategy and assure your target users don’t delete your app before they get a chance to reap its benefits. So give us a call. Let’s get started.

James Hoshor

James is a Senior Mobile Strategist & Solutions Architect for Propelics. He has over 20 years experience in executive leadership, strategic planning, marketing and business development in information technology. For the past 10 years James has worked with many clients across multiple industries, including financial services, insurance, retail and manufacturing, approach mobile strategically to identify and deliver solutions that result in market differentiating solutions and great user experiences.

More Posts

Follow Me:
Twitter