Efficient and Effective Mobile App Testing

A mobile application is a program that is built to be used on mobile devices (smartphones, tablets, and wearables). With the global boom of mobile device usage in the last decade, the scope of mobile applications has also increased enormously. The mobile apps in the market currently perform most of the tasks that were previously restricted to a PC.

But testing mobile applications is not as straightforward as testing web applications. Testing of these mobile apps within the stipulated time, without defect leakage, and confirming if they provide acceptable UI, UX rendering, functional and non-functional behavior across multiple platforms and devices plays a crucial role in deciding the success of every mobile app. The multitude of devices with their various operating systems and specifications has given rise to a wide array of test environments on which the mobile app codebase should be tested in order to confirm if they behave the same.

Apart from the functionalities that the mobile app provides, the users will be more concerned about the UI, UX, performance, compatibility, and reliability of the mobile apps. So the tester should consider those non-functional requirements which are often “unsaid” requirements during testing.

To check whether the application fulfills the expected functions, the tester performs functional testing. Mobile app use cases are designed based on the scenario of daily or continuous application use. The test cases need to be prepared on the basis of business processes and the various user actions that the mobile app allows the end-user to perform within the app and in conjunction with other apps and settings of the device.

Also, it is necessary to test the applications on 2G, 3G, 4G, and Wi-Fi networks. It helps the tester to simulate the performance of the mobile app on different network conditions. Testing the mobile app’s crucial modules without an internet connection is another important requisite. Alerts should be shown gracefully, and data entered should be stored locally, so that the user doesn’t have to enter the data all over again.

Emulators & Simulators vs Mobile devices:

Many of us think these two tools are the same but they are not.

Emulator: It simulates both the mobile OS and hardware, and it is more device-specific. Since it replicates the hardware it helps in finding unexpected responses during testing. One disadvantage of using an emulator is that it does not account for battery stress or depletion. So low battery interruption testing of the mobile app is not possible with Emulator

Simulator: It is faster than the emulators as they only simulate the OS and not the hardware. It helps in the quick exploration of the apps and also in capturing any unexpected behavior.

Real Devices: Ideally there would be no better testing than installing the mobile app and testing it on real devices. We can subject the devices to real-time user conditions such as varying weather, temperature, locations, and networks during the testing. Also, the performance of the mobile apps on real devices will be typically faster and hence it’s easier to test for interoperability.

Considering the above, it’s great if you can test your mobile apps on real devices rather than on emulators and simulators but in the case of non-availability simulators and emulators are definitely a great alternative. The main concern of using both emulators and simulators is that they may generate both false positive and false negative results, which may have an impact on the organization’s ROI.

Mobile Application Testing Process:

Preparation and Strategy: It includes a number of actions including deriving prerequisites, targeted devices identification, finding the app type, etc.

Identifying Testing Types: Involves the combination of manual, automated, and cloud simulator testing along with non-functional testing types such as performance, load, and security testing. This assists in identifying all the potential defects and helps in meeting the objective of the application efficiently.

Test Script and Test Case Design: An exhaustive list of test cases factoring in different elements of the device and the mobile apps are the foundation for performing testing that’s thorough and nears perfection.

Setup of Testing Environment: Setting up the proper test environment plays a vital role to ensure the success of mobile app testing.

Type of Testing: Usability testing ensures that the user finds the mobile app intuitive and easy to use from several different aspects. Performance testing helps in ensuring stability and interoperability. Security Testing discovers the potential security flaws which could be exploited if left unchecked and helps in preventing unauthorized access to the mobile apps. Testing across multiple devices needs to be prioritized based on the percentage of app usage on specific devices, most used devices in specific demography of target customers, market demand, and traffic rate.

Need for Manual Testing: It provides us feedback on UI, UX along with the functionality of the mobile app on different devices and conditions. Factors like performance, battery drainage, or overheating of the device due to the mobile app should be identified early enough through functional mobile app testing.

Need for Automated Testing: It speeds up the testing process and reduces manual regression effort, reducing the cost associated with testing in the later cycles and the time-to-feedback. Functionalities that are repetitive should be handled with automated testing. This is ideal for mobile apps that have a large number of workflows and have a roadmap for several years. Automation improves the extensive test coverages by running thousands of automated test cases simultaneously on multiple devices. Automation tools like Kobiton, LoadStorm (for performance), Xamarin Test Cloud are widely used to automate and run the tests in a cloud-based environment.

Choose Integrated Testing: It is a combination of the above two for maximum efficiency, time, and money-saving with 80% automated testing and 20% manual testing. Combining these two approaches help save up to 70% of the cost. The percentages can be changed depending on the complexity and the concept of the mobile app.

Currently, in the market, we have a large number of Android devices available with different screen resolutions and sizes and there are high possibilities that the mobile app layout and design get distorted in some of these devices. Whereas when it comes to Apple there are only a countable number of iPhones and iPad devices which more or less have similar specifications and in turn, the behavior of the app also remains pretty much consistent. It is the tester’s responsibility to ensure that all images and the UI elements of the app are displayed/aligned properly and function properly on all devices.

Importance of recording logs:

Log analysis is one of the main aspects of performing effective and efficient software mobile app testing. The logs are generated usually when the apps crash or when it does not respond to the user for a noticeable period of time. Crash logs and general app logs are the most commonly used information to debug Android and iOS mobile apps.

As testers, we must have a consummate understanding of the product and technologies used to build the mobile app in order to examine the logs effectively and provide the root cause analysis for the issues we encounter. The ideal way to evade the high impact of such issues is to catch them as early as possible in the software cycle.

Analysis of logs helps in improving the QA process in the below-mentioned ways:

• Helps in analyzing the issues and identifying the possible root cause of the defect. Also helps in improving the reproducibility of the issues as their root cause has been identified.
• Helps in reducing 50% of the developer's effort in reproducing the defect and fixing it.
• Effective log analysis helps in identifying the problematic component and in reporting quality defects.
• Logs could help QA in better understanding the mobile app's functioning and how the code has been written, where it breaks, and which particular action causes it to break.

Android Studio and Xcode simulators are commonly used to prototype, test mobile app builds, and retrieve the logs from the mobile apps.

The platforms have an inbuilt API for setting and launching the application under test in an emulator or real device. So as testers, we need to actively monitor the logs to capture the issues early. To prevent loss of customer base via uninstallations and bad user ratings, it is extremely important to diagnose and rectify every single crash that’s to be found in the mobile app.

Recommendations:

• Cross-platform Interaction Testing is always recommended in order to ensure seamless communication between Android and iOS versions of the mobile app as both use different programming platforms and languages.
• Mobile app size — always keep a tab on the app size after each release.
• App upgrade, ensuring the app doesn’t crash on upgrade and whatever preferences the user had saved in his account with the previous version should be retained when the upgraded version is installed.
• Since many vendors have their own wrappers, the app should support OS upgrades.
• App permission: The app must explicitly ask its users for access to their contacts, camera, gallery, location, etc.
• Be Proactive. Always be ready to share your views. Keep your criticisms constructive and cite how each issue or improvement that you report might be seen in the eyes of the user.

Conclusion:

Scenarios like battery consumption, interruption testing, testing on different networks (3G, 4G/Wi-Fi), testing while switching networks, etc. should be incorporated into the functional test plan for mobile apps.

The attitude of testers is subject to scrutiny especially in mobile app testing as it involves testing the same app on different devices by including all the different features that each of those devices has to offer. The choice of the test strategy, the different types of testing included in it, the appropriate usage of simulators/emulators, devices, and testing tools together decide the effectiveness and efficiency of mobile app testing.

Author: Saikumar. S