Tutorial: Automated testing of 2FA with Mailosaur and Katalon

Katalon is an automated testing platform that runs on Linux, Windows, and macOS. When combined with Mailosaur, you can use it to test and validate SMS and emails resulting from notifications, password resets, two-factor authentication, and other user actions.

This tutorial demonstrates how to set up a Katalon project with Mailosaur and use it to check that two-factor authentication (2FA) or multi-factor authentication (MFA) codes are successfully received. This gives you the tools to automate testing using real SMS messages sent to a Mailosaur phone number so that you can be sure that your app is correctly sending 2FA codes in production.

What you need to automate SMS testing

Whether you are relying on in-house code or an authentication service, it’s critical to test your 2FA and MFA solution because its failure means your users cannot access your product.

Mailosaur is a testing tool for email and SMS that gives you real email addresses and phone numbers that you can use to automate the testing of your messaging, notifications, email login links, password reset emails, order confirmations, and any other kind of SMS or email your app might send.

It provides an API and libraries for popular languages and platforms, so it can be readily integrated with your existing automated testing workflows. This includes Katalon, a widely used testing platform that can test desktop, mobile, and web apps and APIs.

To automate the testing of SMS and 2FA/MFA with Mailosaur and Katalon, you’ll need the following:

• A Mailosaur account with a test phone number. You can start using Mailosaur by signing up for an account.
• A server-restricted Mailosaur API key and its Server ID, created from the Mailosaur dashboard. If you’re testing authenticator app codes, you’ll also need a standard API key.
• A way to send a test one-time password (OTP) to the Mailosaur test phone number. You can use your mobile phone for this!
• Katalon installed on your system. Download Katalon and sign up for an account before proceeding.

How to automate SMS 2FA/MFA testing using Mailosaur and Katalon

Once you’ve successfully signed up for a Mailosaur account and received your test phone number, you’re ready to start automating your SMS testing.

Open Katalon and sign in, then create a new project by selecting File > New > Project from the toolbar. Give your project a name, and set its 'Type' to API/Web Service.

You will then need to add the Mailosaur Java library and dependencies to your Katalon project. Download the latest release of the following .JAR Java package files:

• The Mailosaur JAR
• Google Guava (download the JRE, not the Android version)
• Google HTTP Client (Gson)
• Google Gson

Add them to your Katalon project by navigating to Project > Settings in the toolbar, then selecting Library Management. Under External Libraries, click the **Add **button and open the downloaded .JAR files. Click Apply and Close when you’re done.

In Katalon, the keywords that represent test actions are organized into packages. In the Tests Explorer sidebar to the left of the screen, right-click on the Keywords item and select New > Package, naming it 'mailosaurPackage'.

Right-click on 'mailosaurPackage' in the Tests Explorer, then select New Keyword. Set the new Keyword Class Name to 'MailosaurKeyword', then replace the code in the MailosaurKeyword.groovy file with the following test function:

package mailosaurPackage

import com.kms.katalon.core.annotation.Keyword
import com.mailosaur.MailosaurClient
import com.mailosaur.models.*

class MailosaurKeyword {

    @Keyword
    static String getSmsOtp(String apiKey, String serverId, String phoneNumber) {
        MailosaurClient mailosaur = new MailosaurClient(apiKey)

        MessageSearchParams params = new MessageSearchParams()
        params.withServer(serverId) // Set the Mailosaur server to search

        SearchCriteria searchCriteria = new SearchCriteria()
        searchCriteria.withSentTo(phoneNumber) // Search for messages to your Mailosaur test phone number

        Message sms = mailosaur.messages.get(params, searchCriteria) // Perform message search
        return sms.text().codes().get(0).value // Extract and return the code from the first result
    }
    
}

Here’s how it looks in Katalon Studio:

Katalon Test Cases contain the variables and conditions you use to determine whether your code is performing as expected. To create a new Test Case, right-click on Test Cases in the **Tests Explorer ** and then select New > Test Case, naming it Mailosaur 2FA Test.

In the Variables tab for your Mailosaur 2FA Test, create mailosaurApiKey, mailosaurServerId, and mailosaurPhoneNumber and fill in the Default value, using the values from the Mailosaur dashboard.

In the Script tab, copy and paste the code below under the existing import statements:

// ...existing import statements

import mailosaurPackage.MailosaurKeyword as Mailosaur

String otp = Mailosaur.getSmsOtp(mailosaurApiKey, mailosaurServerId, mailosaurPhoneNumber)
println("Retrieved SMS OTP: " + otp)

Here’s how the code looks in Katalon:

You can see the effect of this in the Manual tab:

From your mobile device, send an SMS that looks like an OTP verification message (for example, OTP 12345) to your Mailosaur phone number, and confirm its receipt in the Mailosaur dashboard. Mailosaur will automatically extract the code from messages that look like verification messages.

You can now run the Katalon Test Case by pressing the green “play” button in the toolbar, and it should pass. Note that by default, Mailosaur will only search messages received in the last hour. The results of your test, or any errors that are encountered, will be visible in the Katalon Log Viewer. You can then continue building your test case for your specific scenario.

How to automate testing MFA codes from an authenticator app

Mailosaur can also simulate an authenticator app for generating and testing MFA codes.

To add this to your Katalon Test Case, enable Authenticator in the Mailosaur dashboard, replace the server-restricted API key in your Test Case variables with a standard Mailosaur API key, and add a new Test Case variable named authenticatorSharedSecret containing the authenticator shared secret.

Then, add the following code inside your MailosaurKeyword class to generate an OTP code:

 @Keyword
    static String getAuthenticatorOtp(String apiKey, String sharedSecret) {
        MailosaurClient mailosaur = new MailosaurClient(apiKey)

        OtpResult otp = mailosaur.devices().otp(sharedSecret) // Generate an OTP code using a simulated authenticator app provided by Mailosaur
        return otp.code
    }

Then add the following lines to the bottom of the Script tab in your Mailosaur 2FA Test Test Case:

String otpAuthenticator = Mailosaur.getAuthenticatorOtp(mailosaurApiKey, authenticatorSharedSecret)
println "Generated Authenticator OTP: " + otpAuthenticator

You can then use MFA codes sent via SMS, or generated using an authenticator app in your tests to make sure they successfully validate.

2FA and MFA testing can make a significant difference to your business’ success, but it’s easy to overlook, leading to frustrated users being turned away from your app.

If you’d like to learn more about what Mailosaur can do for your testing automation, speak to our sales team, check out our documentation, or start a free trial.