Written by Fred Beringer
Top requirements for a successful mobile testing solution
Mobile apps bring a new set of challenges to testing—fast-paced development cycles with multiple releases per week; multiple app technologies and development platforms to support; dozens of devices and form factors; and additional pressure from the enterprise and consumers who are less than patient with low quality apps. Over the past few years, SOASTA has worked with hundreds of customers transitioning from web-only operations to mobile apps who were in need of a mobile testing solution that can scale with their business. Today, we’re very excited to announce a number of new capabilities to our leading mobile testing solution. These capabilities were developed in collaboration with customers as they scaled their mobile operationswith new departments and partners with requirements to build scalable and secure mobile test labs and increase automation. These new capabilities extend our market-leading mobile testing solution and complete our top requirements for successful mobile automation
Working with our customers, we’ve identified and implemented the top requirements required to build an efficient and scalable mobile testing solution.
Rock solid test automation
2 years ago we took the software testing world by surprise by introducing the first in-app mobile testing product. We wanted to reach the holy grail of software testing and provide object-level automation. There was some doubt and pushback…mostly from competitors whose solutions relied on OCR, which was not an option for us, being too fragile, flakey, slow and totally unfit for mobile test automation. With competitors now conceding that in-app is the only way for stable mobile automation, here are our customers’ must-have requirements for enterprise class mobile test automation:
- Developers and testers should be able to write new automated tests within minutes.
- Product must support the most complex mobile apps on iOS and Android.
- Automated tests shouldn’t require maintenance after each new build.
- Product should have a way to address environment unreliability and real device performance differences.
- Product should integrate easily into the development environment and process.
These requirements resulted in SOASTA TouchTest, the leading mobile test automation product on the market today. Here are deeper dives into how we do it.
Easy to learn and use
- 100% visual test creation environment
– No cryptic scripting language required. Easy to learn by anyone, developers and testers
– Easy to read and understand tests. Gesture names are the verbs of the test
– Drag & Drop visual programming constructs (conditionals and loops)
- Out of the box and ready to use test components
– Inputs, prebuilt output, prebuilt intelligent waits, prebuilt validations and accessors.
- Inside the App image validation
– Full screen, partial area and locator based image validations to cover all possible use cases and extend test coverage
Support for complex apps
- Out of the box support for EVERY and most COMPLEX gestures to recreate the real user experience
– Record the user experience on REAL devices NOT on a PC interface using mouse and keyboard
– Gestures and app action supported with precision across iOS/Android, Native, Hybrid and Mobile Web
- Native access to EVERY internal object and UI element of the application
– Native object locators to identify objects and create the most meaningful and robust validations.
– Tests relying on objects validations are fully reusable across different version of the app, devices and OS
- Built-in Intelligent waits
-The best way to handle test flow and environment unreliability
Fit into ALM
- Integration with all major Continuous Integration frameworks (Jenkins, Bamboo, etc.)
-Allows for 100% no-touch automation
– From Jenkins, push and remove apps on the device, reboot and reset the device without any human intervention. Every manual step eliminated for maximum process compression
– Build and setup mobile labs in minutes
– Register devices in seconds and start executing tests on them right away
– Remotely execute tests on tethered or untethered (Wifi, 3G/4G) devices from anywhere in the world
– Mobile labs can be leveraged for functional AND performance testing. No additional 3rd party testing software needed.
What’s new in TouchTest?
As our customers grow their mobile test automation practices, they add new mobile applications to the mix as well as onboard new teams (including offshore teams) to the test automation effort. This onboarding of new apps and new teams needs to happen quickly to keep up with the fast pace of mobile development. You can’t lose a beat! Because SOASTA TouchTest leverages in-app testing, mobile applications need to be instrumented with our library. We today offer two approaches to this instrumentation
- Static Instrumentation
– In this environment, the library is integrated at build time by the development team owning the continuous integration process. SOASTA provides a utility as well as a Jenkins plugins automating this process. The process makes the necessary changes to the iOS or Android project as an automated build task.
- Dynamic instrumentation
– This is SOASTA’s newest approach to integrate the TouchTest library inside the application. Depending on the development environment, it may not be practical to setup a new build specifically for testers, and they might not have the necessary knowledge, skills or access to build an iOS or Android project by themselves. This is why we’ve added the ability for our instrumentation utility to handle iOS IPAs and Andriod APKs directly rather than making any changes to the development project.
With our new dynamic instrumentation, testers can now integrate the SOASTA TouchTest library without any development knowledge.
The process to make apps testable is easy:
- Download our MATT (Make App Touch Testable) utility from the TouchTest Welcome Page
- For iOS dynamic instrumentation: java -jar MakeAppTouchTestable.jar -ipa [Path to the IPA] -signingidentity [name of the signing identity] -provisioningprofile [Path to the provisioning profile] -url [TouchTest server URL] -username [UserName] -password [Password]
- For Android dynamic instrumentation: java -Xmx2g -jar MakeAppTouchTestable.jar -apk [Path to the APK] -androidsdk [Path to the Android SDK] -url [TouchTest server URL] -username [UserName] -password [Password]
The resulting IPA or APK can be now installed on the device with our iOSAppInstaller utility on iOS or via ADB commands for Android, ready to record and replay your tests. It can’t get any simpler for you to get started with TouchTest!
Secure and Scalable TouchTest Mobile Device Cloud
Over the past few years, we’ve seen new requirements emerge for developers and testers to run their mobile automated tests.
Way back in the history of mobile app development (or, still), developers had a local device or emulator connected to their development machine, allowing them to quickly run some of their local unit tests. Pretty soon, they wanted to run these same tests on multiple OS versions, form factors and sometimes, across platforms (iOS or Android). They had to connect all these devices to the build machine to run their tests. Since we’ve introduced TouchTest to them, they started building additional tests (especially integration tests) that had to be run very quickly, in parallel, against a larger pool of devices every night. As the teams grew, additional apps needed to be tested, at different times during the day, and the requirement for true mobile test device management emerged to accommodate this growth. The most important for developers was to be able to run all their tests, really fast and get feedback instantly. Interestingly enough, since they’re used to running unit tests which typically don’t include visual feedback, they’re perfectly fine using a headless mobile device pointing their tests to a distant device to run their tests. All they need to know is that the pool of devices is ready to execute tests.
Teams of testers are more and more dispersed throughout the organization. Sometimes, they are organized in scrum teams working closely together on different functionalities, and sometimes they are testing different sets of apps in more and more distributed across the world with a mix of local and offshore testing teams. In this type of environment, efficiently managing a mobile device pool is critical to avoid any false-negatives in the results of the tests as well as making sure devices are available for the right team to run their tests when they need them. In Enterprise companies, especially where regulatory and security compliance is mandatory, a dedicated private device cloud sitting behind the company’s firewall has to be built and configured.
With these requirements in mind, SOASTA is very excited today to introduce the TouchTest Private Device Cloud that lets developers and testers execute their TouchTest tests in a secure and dedicated environment.
- Dedicated TouchTest Private Device Cloud: In this environment, customer’s devices are securely connected inside a mobile device lab and are managed by the customers. The solution includes an online Mobile Device Management console allowing developers and testers to remotely manage the devices i.e., Install apps, reboot, turn them on and off, as an added benefit, allowing them to remotely interact with them, including manual testing.
The icing on the cake: Remote manual interaction with the device cloud
The customers we work with operate under very different working environments: Some engineering and testing teams are local, some are distributed across different organization within the same building, some are distributed geographically, and some work for different companies in an offshore environment. In all these scenarios, everyone needs access to devices to execute their tests and for the most complex tests (functional, integration and performance tests) they need to access the device for troubleshooting purposes. When something goes wrong with one test and they don’t have access to the device, developers and testers don’t want to spend time figuring out what the problem might be. Is it a problem with the app? A problem with the device itself? Is there a popup breaking the test? Being able to remotely interact with the devices eliminates these questions and accelerates the feedback loop. Being able to remotely interact with devices solves a number of problems that might slow down the testing efforts.
This is the type of problems our customers wanted to solve:
- Troubleshooting automation
- Change settings on the device
- Collaboration i.e., Being able to show the device to someone remote for feedback on a design or behavior
- Manual testing of a particular feature on a specific device that is not available locally
These problems are important for our customers and we’ve taken this important step to help solve them. This is why we’re excited to add dedicated private device cloud capability to our platform! These clouds integrate seamlessly with TouchTest to offer the perfect solution for mobile test automation! Plug them in, create your tests, and off you go!
You can read more about our mobile testing solution HERE.
-To learn about the best practices in improving speed, quality and responsiveness in Mobile Development go HERE
About the Author