Virtually every organization we speak with is exploring the best way to migrate their applications from the desktop and web to a mobile environment. In fact, much of our testing and consulting work these days is helping companies test their apps in a mobile environment. Through this work, we have learned a number of testing best practices for this unique space.
Mobile Testing
From an execution perspective, many testing firms consider mobile testing a non-brainer. It is not. Testing mobile apps is fundamentally different than other types of testing. It cannot be handled like a desktop testing project due to the significantly higher business risk associated with operating in a mobile-enabled world. A significant amount of revenues and goodwill is dependent on how the small app icon works on a smartphone or tablet.
To wit, a recent survey of 2000 U.S. adults found that “almost nine out of ten Americans associate negative feelings with brands that have poorly performing websites and mobile apps.” Furthermore, the app must-
- Work when they need it – 72%
- Return correct “trustworthy” results – 45%
- Have fast performance and load times – 62%
This feedback suggests that consumers have higher expectations for a company’s mobile apps than their online or bricks & mortar operations. As a result, organizations need to ensure their mobile apps are hitting their performance, trust and user interface objectives. One way to know this is to check out Applause.com, a mobile app analytics tool, to see if you are disappointing users in any of the above areas. If the apps are underwhelming users, its time to revisit the testing approach.
When launching new apps or fixing existing ones, IT managers need to consider these five critical differences when developing their test strategy and plan:
1. Desired Consumer Experience
- How will the user interact with the device? Users can interact using voice activation, typing, GPS coordinates or moving the device, making testing a real challenge.
- What kind of experience do you want to create? Of the respondents surveyed above, 38% said they want an app that was fun and easy to use.
2. Supported Devices and Platforms
Which devices and platforms should or can be supported? The fast-changing mobile sector features a plethora of brands (Apple, Samsung, Blackberry etc.), devices (tablets, smartphones) and platforms (Android, iOS, Windows, Blackberry). Given this complexity, it may be practical to develop a testing plan along the lines of graded browser support used by Yahoo to ensure that the major devices and platforms are supported.
3. Fit/Form Limitations
How do you guarantee performance when you deal with non-standard, small displays and interfaces? Mobile testing must cope with how different applications handle limited screen sizes and interface “ergonomics”(e.g., number of buttons available per screen to navigate to perform key tasks) – in physical configurations that change regularly.
4. Application Types
How do you conduct testing when applications types are coming together in one place? It is not uncommon now to see HTML5, Native and Hybrid application types in one device. As well, testers must understand that testing hybrid apps is very different from native application testing.
5. Mobile Specific Issues
Session management – Mobile devices feature many types of interruptions, utilized on a regular basis. How does the app handle and react to an interruption like a call, push notification or SMS?
Configuration changes – Mobile apps are added, removed and upgraded more frequently than desktop apps. This goes for OS and platform changes as well. How will the app handle the periodic upgrades and future changes while maintaining backward compatibility?
Emulator dependencies – Unlike the desktop, smartphones and tablets rely on emulators and simulators to represent the applications. Mobile testers must have a solid understanding of the strengths and weaknesses of various emulators to deliver effective testing.
Maintaining Security – Many applications dealing with personal data such as a phone book, video or pictures are accessible to other programs. Mobile testing requires an extra level of diligence to ensure that many applications are not making personal data available to others or unintentionally accessing private information in its operations.
Network/carrier differences – Most consumers will experience a variety of network conditions such as 3G, 4G, signal, no signal etc., depending on physical location and network reliability. Furthermore, each device can operate differently depending on the carrier. How is testing impacted by network and carrier differences or the many variants of real-life usage?
Mobile computing is no longer something happening in the future. Having this capability today is crucial to the financial and brand performance of most public and private organizations. Firms can improve the odds of mobile success by ensuring their apps are tested properly and perform to specifications and user expectations. To do this, it is always sensible to partner with proven mobile testers who have extensive technical and industry expertise.