The Automated Mobile Application Testing Tools Market Explained

Selecting and evaluating a mobile application testing tool to support functional test automation can be a cumbersome task. Every tool in the market has a different set of features, limitations, challenges and caveats (which are often not discovered until after purchase or during a POC).

Before surveying the various tools in the market, begin with defining internal requirements for an automated mobile app testing tool. I recommend reviewing our article on the most common mobile application testing automation tool requirements, which is derived from requirements expressed by many of our clients. You will likely find that they align directly with your organization’s automation tool requirements, or that there are some gaps to be uncovered. Once the requirements are defined, evaluating the various tools in the market becomes a manageable task.

If your organization needs guidance determining the best tool to fit your automation needs, we can help!

This article will focus on tools that integrate directly with HP UFT/QTP. HP is a leader in the enterprise market for functional test automation, performance testing and application lifecycle management.

Mobile Automation Software Companies vs. Mobile Device Rental (with Automation) Companies

The market is comprised of two types of companies:

1. Mobile Device Rental companies which specialize in renting mobile devices and provide a mobile automation toolkit (as a subset of the business), and
2. Mobile Automation Software companies which specialize in developing software solutions explicitly designed to support mobile automation.

Mobile device rental companies generate a vast majority of their revenue from device rental contracts, either paid on an hourly basis or based on setup and renewal fees. These companies provide public (and private) devices which are consumed by the organization performing functional tests (automated or manual). These companies entered the mobile device rental market with a focus on manual testing.

The mobile automation tool which integrates with HP UFT/QTP (as an add-in) is a subset of the business which helps to grow the primary business model (the managed mobile device lab, i.e, “the public/private cloud”). This allows an organization to focus on testing a wide range of mobile devices and mobile operating systems without the need to self-manage an internal lab. Mobile devices and/or operating system versions which are less commonly used by end users can be tested without the need to purchase a device in-house.

Organizations with a very diverse set of supported mobile environments, without data confidentiality and security concerns, are usually a good candidates for a managed mobile device lab model (which is contracted out to the device rental company).

Mobile automation software companies focus on delivering software that integrates directly with HP UFT/QTP (as an add-in) and they typically do not rent hardware devices in a managed environment (i.e., “the public/private cloud”). The mission of the company  is to develop high-quality software solutions that support mobile application automation best practices. These companies excel at this and they typically support all or most of the mobile app automation best practices and requirements we’ve covered in this blog.

Management of a mobile device lab is in the hands of the organization (the same organization which is testing their internally developed mobile application). With this in mind, it is noteworthy to mention that some key players offer a solution which allows for an easily self-managed mobile device lab (i.e., a “private cloud”).

Organizations that work with highly confidential data, have compliance restrictions or high security concerns are good candidates for the self-managed mobile device lab model. They have complete control over all aspects the mobile device lab.

The Mobile Device Rental Market and Vendors

Key vendors in the mobile device rental market are Perfecto Mobile (“UFT Mobile”) and Keynote (DeviceAnywhere). The technical means in which mobile devices are interacted with are different, but the end result is mostly the same.

DeviceAnywhere taps into the hardware layer of the mobile device. In a nutshell, they dismantle the mobile device and “hook into” the device PCB board to intercept and relay hardware commands and video. This results in jailbreak-free and root-free devices because they interact with the hardware layer. DeviceAnywhere is a private cloud solution.

Perfecto Mobile utilizes a USB based connection to issue commands for hardware and software interaction. To relay video, VNC is used for iOS devices and USB or a physical camera is used for other devices. Due to the relaxed security measures of the Android operating system, Android devices are generally not rooted.  iOS devices require a Cydia application to use VNC, which indicates iOS based devices are jailbroken. The Perfecto Mobile “source-code instrumentation” product is in the Beta stage (at the time of writing), which allows for iOS devices which are not jailbroken to be tested. “UFT Mobile” is both a private and public cloud solution.

Read more about why jailbreaking and rooting mobile devices for testing is not considered best practice.

There are challenges associated with automating mobile devices since true object-based recognition is not supported, as well as many of the other automation best practices for which HP UFT/QTP is well-know. Both companies currently support image-based and OCR-based objects (screen scraping for text), which are generally utilized by automation engineers only if true object-based recognition is not an option.

As previously mentioned, integration with HP UFT/QTP for test automation is a subset of the primary business for mobile device rental companies. For this reason, mobile automation best practices aren’t widely supported.

The Mobile Automation Software Market and Vendors

Key vendors in the mobile automation software market are Mobile Labs (Trust), Jamo Solutions (M-eux), and Experitest (SeeTest). Without focusing on the vendor specific details and feature support, these companies deliver software which integrates directly with HP UFT/QTP (as an add-in) and most support the best practices of mobile functional test automation including:

• No device jailbreaking or rooting requirement
• True object-based recognition
• Physical device, emulator and simulator support
• Native, hybrid and web application support
• High re-usability of scripts and test assets
• Common device functionality support (i.e., touch events)
• Integration with performance testing tools

Since these companies focus on delivering a feature-rich mobile automation software solution, they have the most features that support automation best practices and ease of scripting. If an automation engineer is experienced with HP UFT/QTP, they simply create mobile test scripts using existing skill-sets and the same automated test creation workflow used for other applications.

Simply put – automation engineers do what they have always done.

The Best Mobile Automation Tool

Selecting the best tool for your organization depends on the requirements established by the organization. Both mobile automation software tools and mobile device rental companies have their role in the mobile testing automation market.

We can help your organization determine the best tool based on defined requirements, or we can help define the requirements based on your needs. The best part is, we can help cut through all of the marketing material and sales presentations and get straight to understanding your needs and guiding you in selecting the best mobile automation tool for your organization.