December 11, 2017

Common Email QA Edge Cases You Need to Be Aware Of

Justin on December 11, 2017 in Technical Articles

There’s never enough time to test everything and it can be challenging to prioritize what needs to be covered. Edge cases are often neglected at the test-driven development (TDD) phase, which means that quality assurance (QA) teams are usually tasked with pushing features to extremes to see where they break. While you can’t anticipate every scenario, there are some common email edge cases to be aware of when QA testing.

Email Addresses

Most users have fairly straightforward email addresses, such as a Gmail or corporate account, but there are many edge cases that can cause issues. Developers will often use regular expressions to validate that an email address is valid, but unusual email formats can fool many commonly used regular expressions and lead to a host of problems.

Examples of unusual email addresses that should pass validation include:

  • Custom top-level domains (e.g. name@domain.name)
  • Quotes around email addresses (e.g. “name”@domain.com)
  • Brackets around IP addresses (e.g. name@[123.123.123.123])

Examples of unusual email addresses that shouldn’t pass validation include:

  • Invalid IP addresses (e.g. name@123.123.123.1234)
  • Multiple dots (e.g. name..name@domain.com)
  • Encoded HTML (e.g. Name <name@domain.com>)

This blog post on Microsoft’s MSDN provides a comprehensive list of these unusual edge cases that you may want to consider testing to ensure the best user experience. It’s also helpful to have a cheat sheet or Selenium script on hand to test these various email formats to ensure everything is working properly without having to remember these formats every time.

The good news is that, once the right regular expressions are used in the development stage, these issues should cease to be a problem in the future.

Email Formatting

Most users have common email clients like the Apple iPhone Mail App or Google’s Gmail, but there are several edge cases that can cause problems. Depending on your users, you may also have to test email formatting on legacy email clients or uncommon mobile devices.

According to Litmus, the Apple iPhone was the single most popular email client with a 33% market share followed by Gmail at 20% and the Apple iPad at 13%. This means that all emails should look good on mobile devices like the iPhone and somewhat unusual sizes like the iPad. For example, users shouldn’t have to scroll to the side on an iPhone to see a button that must be clicked to complete a registration process or make a purchase.

But, enterprise users often have use of some more unusual email client software. For example, older versions of Microsoft Outlook or Blackberry mobile devices may be standard issue within an organization, which could make it important to test all email clients.

Companies like PreviewMyEmail, EmailOnAcid, and Litmus give you the ability to preview emails on multiple devices and software versions to ensure that they are properly formatted, which is often much easier than manually testing on physical devices or simulators.

Other users may prefer plain text versus HTML emails, which means that it’s important to test both types to ensure that they’re properly sent and formatted across all of these devices.

User Metadata

Metadata may play a role in transactional and marketing emails. For example, transactional email software like MailGun lets developers set recipient variables that enable personalized emails. Most QA engineers are familiar with common variables – such as a user’s first name – but there are many other variables that may influence the content of an email.

For example, suppose that you have an ecommerce application that supports multiple currencies depending on the browser’s location at the time of sign up or purchase. If everyone in the QA team is in one location, it’s easy to forget to test to make sure that emails generated contain foreign currency prices and symbols.

The most common way to test location-based edge metadata is using a virtual private network (VPN) when interacting with the app. For example, a U.S.-based QA team may connect to a U.K. server to simulate being in the U.K. when an order is placed. They can then check the resulting email to ensure that the invoice or receipt is priced in pounds rather than dollars. Some browsers, like Chrome, also have geolocation extensions for these purposes.

Other metadata may be generated from certain actions. For example, an ecommerce application may store order value information and display different images in a marketing email based on that information. It’s a good idea to discuss these meta data points with the development team to ensure that the QA process encompasses all of these scenarios.

Email Actions

Transactional emails often contain the ability to take many different actions. For example, a new user might receive a welcome email that contains a confirmation link and must be followed to complete the account setup. Most QA teams will test the confirmation link to make sure it’s working, but it’s easy to forget about other links in the same email, such as an unsubscribe link or an account management link.

The easiest way to automate the testing of email actions is using a service like Mailosaur, which lets you access these links via an API. If you’re using a testing automation framework, like Selenium or Capybara, you can use this API to automatically click each link in an email and ensure that the correct result occurs (e.g. the right resulting webpage coming up). This is much easier than managing individual email addresses or setting up your own testing SMTP server.

Other transactional emails may have the option to reply to take an action. For example, the application may unsubscribe a user when they reply to the email with the word ‘unsubscribe’ in the body of the email. It may be important to test these features to ensure compliance with CAN-SPAM laws and ensure the best user experience.

The Bottom Line

Quality assurance testing involves a lot of tradeoffs since there’s never time to test everything, but there are many common edge cases worth considering. Email addresses, email formatting, metadata, and email actions are four important categories of edge cases that may affect a high number of users. By keeping these tips in mind, you can ensure that you’re covering all the bases and avoiding any problems.