Transactional and marketing emails are alive and well in 2021, with product teams betting big on email at a time when other communication strategies are either inefficient or simply unworkable.
With the increased focus on email automation, teams are running into the usual email pitfalls: their transactional or marketing messages are still hitting recipients’ spam folders, or being deemed unreadable for formatting reasons. In other cases, emails don’t even make to the intended inboxes, due to simple (and easily avoidable) typos in the recipient emails.
What if there were a solution to these common issues?
Email testing tools, which allow you to conduct test automation, spam analysis, and much more, do just that.
What kinds of email testing tools are there?
There are several types of email testing tools, based on the primary function of each. One category includes SMTP test tools to identify and diagnose any SMTP server issues that may be preventing your emails from reaching your intended recipients. By verifying that your login details are correct and that there are no threats to your SMTP server, these tools help to keep your email safe and thereby increase your email deliverability.
Message viewer tools, on the other hand, allow you to mantain your email design and formatting, even across multiple platforms. By viewing emails in the way your prospect or customer would, you can catch mistakes in your design and ensure that all elements are properly displayed. You can rest assured that only your best, finalised version is going out to recipients.
Now if your emails are not getting much of a response or if you simply aren’t getting recipients to complete their registrations due to a missing confirmation email, you might have a reputation issue on your hands. In these cases, you will want to use a tool to check your Sender Score. Too low a score can mean that your emails are being classified as spam. The right tool will let you know the calculation behind your score and provide actionable insight to get you in the green.
While there are a host of other types of email marketing tools, including ones focused on subscriber verification and subject line testing, our point is simple: to use email effectively in your product in this day and age, you need to be familiar with them all.
Why are email testing tools useful?
For developers, email testing tools make it easier to spot and fix issues with emails, from design to automation flows, regardless of your customers’ email client or platform. It’s common to spot email issues only when customers tweet at your company account with a screenshot of a typo, a broken design, or an email that the customer has opted out of. By testing your emails thoroughly you reduce the chance of issues getting surfaced and, at the same time, make the customer experience more consistent.
Many organisations choose to do email validation within their unit tests, but this approach doesn’t cover a whole set of issues around deliverability, SMTP server configuration, changes in how Gmail handles its automated folders, and how emails are displayed in various email clients. To catch these larger issues you have to replicate a path that an email would take in a production scenario, and this is exactly what the email testing tools help you achieve.
Now that you’re aware of a few categories of email testing tools and are familiar with why they are valuable, let’s look into a few specific tools you can get started with right away.
Examples of email testing tools
smtp4dev
Smtp4dev is an open-source testing tool that allows you to set up a fake SMTP email server without any special configuration. By testing emails from the development and staging environments, you can avoid emailing real recipients. The tool uses a simple GUI and shows necessary details for development.
MailCatcher
MailCatcher is a testing tool that runs a simple SMTP server to receive and display messages in a web interface. Similar to the above tool, it allows you to inspect email contents in a development setting rather than sending it to actual recipients. With MailCatcher you can download the email to view it in various mail clients. You can list attachments and download them in separate parts and open any links in a new window.
While MailCatcher has a simple installation process and a simple GUI, make sure to pay attention to dependencies to ensure the tool works correctly.
MXToolbox
If you’re in need of a server diagnostic tool to identify real-time issues, you can try out MXToolbox. The tool offers both paid and free solutions that allow both DNS monitoring for email and the ability to look up information necessary for troubleshooting.
For example, if your customers are not receiving your emails, you can perform a blacklist check to ensure your email has not been blacklisted to reduce spam. Additionally, you can check your email health to ensure that your systems are reaching optimal performance.
Mail-tester
To get the spam score of a particular email message, mail-tester requires you to send a test email to a randomly-generated address. It then checks your message contents, mail server, and sending IP to provide you with a detailed report in addition to a numerical value out of 10.
The report breaks down individual criteria and shows where you lost or gained points. This includes the ability to view your message, see whether your email might be considered spam, whether you’re properly authenticated, and if your message could be improved. Having feedback for your message allows you to tackle each individual issue and improve your overall score, thereby your increasing the deliverability and open rates.
Do keep in mind, however, that even with a perfect score, your emails could still be undeliverable due to the various types of spam filters that different companies use.
SpamAssassin
Last, but not least, SpamAssassin is open-source software that uses spam detection techniques to carry out a spam analysis. Using a numerical value, it shows you the likelihood of your email ending up in your recipients’ spam folder. As a sender, your goal is to ensure an email score below 5.
This open-source tool may be difficult to set up due to its complexity. If you decide to use it, make sure to follow the official documentation closely.
Mailosaur - exploratory and automated email tests!
As a software or QA engineer, you’re spoilt for choice when it comes to email testing tools. There are excellent open-source solutions on the market, but your best bet may be a managed service that combines all of the functionality that you need into one easy-to-use platform.
Mailosaur is just that! Our service lets you accomplish things like automating your email tests for user signups, setting up virtual SMTP servers, and viewing your messages just as your recipients would.
Try Mailosaur for free today and start building products that work!
And finally, beware of regex hell
If you’re just getting started with email testing, you'll probably come across regular expressions on QA sites and forums that can help you work with email content, such as links found in the body of an email.
Because you can easily set up a regular expression (regex) using your programming language’s built-in regex primitives, for example, Python’s re module or the global RegExp object in Javascript, it is very tempting to create a series of expressions to use in your tests.
While it sounds like a simple approach, rolling your own logic to pull out and test content is not a sustainable solution - trust us, we tried! You first have to come up with useful regexes that cover all possible cases that, when you learn more about email and all its quirks, quickly become very complex. Second, you’ll need to update the regexes often and test the expressions themselves. Third, some regexes can end up using a lot of computing power at scale and put a strain on your build servers’ CPU, so performance tests on your chosen expressions are essential.