If you’re developing an iOS application, you probably have an idea of the type of user interface tests you need to perform. But how can you get all of your UI tests done in a time-efficient manner? Many manual testers understand the benefits of automated testing but are not sure where to turn. Test automation makes it possible to perform tests with software, reducing the need for a tester to perform the tests themselves.
In this article, we discuss the importance of automated UI testing for iOS apps, and provide recommendations on which frameworks you should use to create your test scripts.
What is iOS UI Testing?
On Apple’s iOS, watchOS, iPadOS, and tvOS operating systems, user interface (UI) testing involves the verification of an application’s behaviour based on a user’s interaction.
For instance, does your iOS application end up in the correct state when a user taps a button or tilts the device? Is the right information displayed on the screen? Does the application crash or give an incorrect output? Answering these questions allows you to ensure your program has minimal issues at launch.
Automated iOS UI testing vs manual testing
Many iOS development teams only perform manual testing on their app, like installing a development build on their own device. There are some notable advantages to automated iOS UI testing over manual testing.
First, automated testing is a repeatable process. You can build a test script to evaluate a specific interaction and use that same script to retest as many times as needed. A computer will run your test scripts in a repeatable manner, with little to no inconsistencies between tests. This approach is optimal for catching errors in your app.
Next, having a computer perform automated testing allows you to scale the number of features in your iOS app without increasing your QA staff. Once your QA team writes a test, they’ll be free to start creating tests for another feature while the computer does its job.
Some scenarios are hard to test, such as email signups and password resets. Tools like Mailosaur can help alleviate these difficulties by providing email addresses or SMS numbers to which your app can send signup verifications and password reset information for testing. Luckily, when you write automated tests, integrating additional automated testing tools requires little effort.
Manual testing is still needed in areas where automated testing cannot go, such as an app’s design elements or colour scheme. These types of tests require human feedback to ensure that an app looks as good as it functions.
Selecting the top test automation frameworks
Our list of top test automation frameworks only includes frameworks that are entirely free to download and use. There should not be any barrier to any of the framework’s available features. Frameworks must also be actively maintained to ensure compatibility with the latest versions of iOS.
Top five frameworks for iOS UI testing
With these two criteria in mind, here are our top five frameworks for iOS UI testing.
XCUITest
XCUITest is Apple’s own free-to-use test automation framework for testing apps on Apple devices. The company launched XCUITest as an extension of the XCTest unit testing framework. The framework is included with the Xcode IDE and does not need to be downloaded on its own.
Because XCUITest is specific to UI testing on iOS, the framework performs the task exceptionally well. Thanks to its focus, XCUITest can execute tests quickly, reliably, and in a stable fashion. XCUITest gets support for all the newest iOS UI features very quickly.
XCUITest does have limitations when it comes to writing test scripts. You can only write test cases in Swift or Objective-C, and not the more popular programming languages like Javascript or C# that other frameworks support. This is not an issue for iOS developers, but can be a challenge if you have non-iOS team members that need to contribute to the test suite.
Appium
Appium is a free-to-use and open-source test automation framework specific to mobile applications. Appium is able to execute tests on not only iOS but Android as well. This enables testers to use one codebase to create scripts for both platforms.
Developers and QA engineers can use any number of popular programming languages such as Java, JavaScript, Perl, Python, and C# to write tests in Appium. But the framework’s reliance on coding makes it complicated to use for entry-level developers.
Appium launched in 2014 and has gained a large community of followers since then. Their Github repository alone has over 14,000 stars. The framework sees releases every few months to keep up with browser updates and changes within iOS.
Detox
Detox is another test automation framework used to evaluate test scripts. The framework supports cross-platform tests, and is usable on both iOS and Android operating systems. While Detox allows testing on actual Android devices, developers are limited to running iOS tests on a simulator at present. The Detox framework supports JavaScript for test creation.
This open-source, free-to-use framework was developed by Wix, a web development service better known for its website building platform. Detox has over 8,300 stars on Github, showing that thousands like the project.
Plus, without reliance on a specific test runner, QA engineers can use whichever test runner they want.
EarlGrey2
EarlGrey2 is Google’s automated UI testing framework for the iOS platform, combining the original EarlGrey with Apple’s XCUITest framework. Similarly to XCUITest, EarlGrey2 enables developers to write tests in either the Objective-C or Swift programming languages.
EarlGrey 2.0 utilises Apple’s Xcode IDE for test creation and execution. To set itself apart from XCUITest, EarlGrey2 waits for your application to be in an idle state before starting tests. This ensures that you will receive the same test results from run to run, eliminating flaky test data. EarlGrey2 also has a set of matchers for UI interactions, and these specialised matchers make UI tests easier to write compared with EarlGrey2 compared to using XCTest’s default assertions.
TestProject
TestProject uses a downloadable Agent alongside their test recorder to execute automated UI tests on iOS applications. The TestProject framework is compatible with both iOS simulators and actual iOS devices. To test on an actual iOS device, you will need an active Apple Developer Account.
The framework supports testing on both Android and iOS, giving testers the opportunity to work with either operating system. Testers can even perform iOS test automation on Windows devices without the need for macOS or Xcode.
TestProject stands out as an automated testing platform because it does not require code to create and run tests. By recording a test, you can use TestProject’s user interface to build a step-by-step test script that the framework will then automate.
Which framework is right for you?
Depending on the specific purpose of your application, some frameworks may suit your purpose better than others.
If you’re building an iOS application and have no intention of taking your app cross-platform, XCUITest is a great choice if you prefer writing tests in Swift or Objective-C. If you find XCTest’s matchers limiting, EarlGrey2 is the next-best option for iOS projects.
But if you prefer JavaScript, you’re better off using Appium, which also works well for getting other non-iOS team members to contribute to the process. Should you desire to take your app across platforms, Appium supports automated testing on Android devices as well. Appium even makes it possible to use the same tests on both platforms with minimal adjustments.
If you find tests written with Appium to be too flaky, consider trying out Detox as it can help reduce the occurrence of flaky tests in hybrid iOS and Android projects.
If you wish to create test scripts while staying away from code, you should consider TestProject. The platform’s test recorder allows you to build tests using points and clicks instead of actual code.
Automate password reset verifications and more with Mailosaur
In this article, you learned about the top five iOS UI testing frameworks currently available: XCUITest, Appium, Detox, EarlGrey2, and TestProject.
You can use any one of these frameworks with Mailosaur to see how your iOS app sends an automated response to any number of email addresses or mobile numbers. With Mailosaur, you can see the exact information your app is sending to each email or number.