Spam filtering is a vital part of the email delivery pipeline. Still, having your carefully crafted customer email end up in the spam folder is a real bummer. Read on to learn all about email deliverability tests, and how to make sure that your transactional emails end up where you want them: in your recipient’s inbox.
What is email deliverability testing?
You know that slightly embarrassing feeling when you have to ask your recipients if they’ve “checked their spam folder” upon your learning that they haven’t received any of your email notifications? That’s the kind of situation we can avoid by checking our email deliverability.
Note that deliverability isn’t the same as the delivery rate. The latter describes the ratio of emails that were accepted by your addressees’ email servers and didn’t bounce.
Deliverability (otherwise known as inbox rate) determines whether a message is likely to be classified as a legitimate email, rather than being relegated to the spam folder. Therefore, when we perform a deliverability test, we’re able to verify that our emails will reach the inbox as intended. To understand how we can improve our own deliverability rates, let’s look at the inner workings of a spam filter.
How do spam filters work?
Email ‘spam’ got its name from a Monty Python sketch of the same name, in which every meal is accompanied by an increasingly generous serving of spam. Like the tinned meat in the sketch, you can’t escape unwanted commercial emails. Estimates of the global annual spam rate vary, but it’s reported to be somewhere between 45 and 73 percent. That’s roughly half of all emails sent worldwide, or more.
Catching spam before it reaches the user has therefore always been at the top of email providers’ agenda. Email spam is not only annoying and takes up precious storage space, but it also poses serious real-world dangers. Having your identity stolen through a phishing campaign or contracting a malicious computer virus can cause you significant harm.
But the biggest difficulty in separating spam from legitimate email is the issue of false positives, which must be kept to an absolute minimum. In binary classification problems, false positives are data points that get flagged as something they’re not. Imagine waiting impatiently for an email from your customer or friend, only to find out that it’s been flagged by a spam filter! A mistake like that might well compromise your relationship.
For these reasons, providers work hard on highly accurate spam filters that combine hard rules with probabilistic methods. An example of a rule-based filter is the use of blocklists to record suspicious IP addresses and domains. Checking whether you’re on such a list is part of every email deliverability test. Interestingly, if you’ve been placed on a blocklist, simply creating a new domain from which to send your bulk mail doesn’t solve your problem. A new address first has to prove its good intentions before being regarded as trustworthy by the providers.
While spam checkers will never disclose all their rules (after all, that would give spammers exactly what they want), there are some general patterns in spam detection. A filter will check the email’s content: are links genuine? Is HTML well-formed? Does the text use a suspiciously high number of words regarded as “spammy”? But the most valuable details are found in the part of the message normally hidden to the user: the email header contains detailed information about senders, transmission routes and authentication protocols, all of which aid spam filtering.
Spam regulations
Over the past years, it’s become significantly easier to unsubscribe from unwanted mailing lists. This is not necessarily due to the benevolence of email marketers: not only does the lack of an “unsubscribe” button impact your deliverability, but omitting it is also unlawful in certain countries.
Australia’s Spam Act, for instance, requires that a sender make it “easy to unsubscribe” (and that they consented to receiving emails in the first place). Likewise, the UK’s own Data Protection Act prohibits sending emails to individuals who haven’t actively given their consent. The United States CAN-SPAM Act, however, allows companies to send ‘cold’ (i.e. unsolicited) emails.
In order to avoid having their own domains abused for spamming purposes, email providers also provide rules against sending mail in bulk. The Swiss-based provider Proton, for instance, prohibits users of their free plan to address more than 100 people at a time. If you’re a new user, that limit is even lower.
How does email deliverability testing work?
Having learned about the complexity of separating spam from ‘ham’ (not-spam), you won’t be surprised to hear that testing deliverability is actually pretty hard, and involves finding solutions to many different problems. Let’s look at three elements that are essential to deliverability testing: checking an email’s content, its header, and setting up a fake SMTP server.
Content
Whether your email talks about high amounts of cash, addresses your customer as a “dear friend”, or enthusiastically mentions Oprah Winfrey, spam filters don’t like it. However, only doing one of those things likely won’t be enough to earn spam classification. Rather, spam likelihood is assessed by running a vast number of tests on the header and body of your email.
The most commonly used filter is Apache’s SpamAssassin. Since it’s an open-source technology, you can check out all of its rules here. SpamAssassin is customisable and includes a Bayesian classifier (a machine-learning technique) that you can train with your own data. Naturally, it also performs the test discussed earlier, such as checking blocklists and flagging faulty HTML (it turns out that many spammers are sloppy coders). Performing these content checks will increase your deliverability.
Message authentication protocols
Although it’s become significantly harder, the relative anonymity of the internet still lets people pretend to be somebody they’re not. This is known as spoofing. Spoofers manipulate an email’s header to make it seem more legitimate. Usually, they’ll have the ‘sender’ field display the name of a trustworthy individual or organisation to cause you to click on the email without thinking twice. Luckily, well-intentioned senders can prove their legitimacy by using one or several identification protocols.
With the Sender Policy Framework (SPF), a domain admin may create a list of IP addresses authorised to send emails from the domain. The SPF information is added to the domain’s DNS record and may be inspected by any potential recipient to determine whether a sender’s IP address has the green light from that domain.
The idea behind the DomainKeys Identified Mail (DKIM) protocol is similar. DKIM ensures that an email is being sent by an authorised address and the header hasn’t been tampered with along the way. DKIM generates an encryption key that the recipient can use to check the validity of an email.
Finally, there’s DMARC (short for “Domain-based Message Authentication, Reporting and Conformance”). This is another protocol that combines the functionalities of both SPF and DKIM. It also allows the recipient to report back to the sender about any activity from their domain.
All three authentication measures serve to verify your authenticity as the sender of your emails. Making sure that they’re set up correctly will greatly help your messages pass a deliverability test.
SMTP details
In theory, testing an email should be as simple as sending it to one of your other email addresses and checking whether it goes to the inbox or spam filter. In practice though, repeatedly sending malformed emails will likely impact your deliverability and might even result in your sending address being blocked by the providers. That’s why it’s common to set up a dedicated SMTP server for testing purposes.
The Simple Mail Transfer Protocol is responsible for transferring your message to a server and transmitting messages between servers. Your fake SMTP server has all the functionalities of a real server, but you won’t really be engaging in any email traffic. You’re just simulating it to check how well your emails would perform in the wild. Crucially, a fake SMTP server frees you to run your tests over and over again, without your having to worry about deliverability.
Why is email deliverability testing valuable?
Before you start sending out transactional emails to clients or prospects, you’ll want to make sure that they don’t go straight to spam. Even if your recipients are thoughtful enough to check their spam folders, you still cannot afford to have your content be classified as junk email. Not only does spam email frequently go unopened, but getting flagged for spammy behavior will eventually impact your overall sender credibility.
As spammers continue to innovate ways to trick the filters, spam checkers in turn have to adapt to their adversaries. This leads to ever-changing spam compliance guidelines. It’s crucial to have a workflow in place that lets you test your emails continuously and thoroughly before you send them out into the real world. Automated email testing with Mailosaur makes that possible.
Get started with Mailosaur
From managing an SMTP server and juggling blocklists, to checking for trigger words and well-formed HTML: deliverability testing is complex. At Mailosaur, we specialize in taking that burden off your shoulders. Let our fully hosted testing service manage your entire pipeline for you.