How to Test Email: A Personal Journey

What aspects of an email can you test?

You can test ALL aspects of your email and here’s a quick list:

Email deliverability
SPAM score
Headers
Technical tests (DNS authentication, inbox placement, etc)
Email components and interaction elements

Also when testing, a structured approach is crucial, or you’ll get lost in all the data. Worse yet, you’ll waste a lot of time testing the wrong things.

Deliverability testing

Deliverability testing is the first thing to do because it’s about making sure your emails hit the inbox, not the spam folder. And my dear colleague Piotr wrote a great guide on “Email deliverability: Everything you need to know”.

We even got an entire cheatsheet playlist about it. Hit ▶️below.

Anyway, moving onto the nitty gritty of deliverability testing.

HTML check

The HTML check is crucial in email content testing. Since this is not only about different content elements working but also about rendering. If the rendering is bad, the customer will think you are a spammer – spam complaint = sending reputation drop.

Note that I’ll be covering sandbox tests with Mailtrap to check emails before sending them. And that’s without spamming real recipients.

Try Mailtrap Testing

If you want to give Mailtrap Email Testing a try, hit the button above ⬆️to sign up, then follow this setup guide. You be testing your first email in minutes and there’s a free email testing plan.

Later, I’ll talk about production email tests (with Mailtrap and otherwise). You can safely run these with a preselected cohort of your subscribers. For example, production email tests are A/B tests and specific aspects of trigger point tests like email heat maps.

Finally, it’s time to move to WHATS and HOWS.

The HTML check takes a deep dive into various elements (HTML and CSS) of your email template and assesses overall market support across different email clients, and different devices; covering web, mobile, and desktop.

There’s a neat “Only” button, which pops up when you hover left of each provider. Hitting it allows you to see the support for only the client you select. And if you’re dreading deliverability to Apple Mail, for example, it’s a cool trick to see how to fix your template so that Apple loves it.

On that note, I’ll move on to show the other email template elements you can test under HTML Check.

Email body

The <body> element section shows you the support for the body of your email template. Note that this isn’t body copy of an email. It’s the HTML body of your email and how deliverable it is across different clients.

In this example, you can see that there’s partial support for the <body> element and the problematic area is on line 6. When I click 6, next to “Found on lines:”, I’m taken to the HTML Source tab, and I can immediately see the problematic line, so it’s straightforward to fix it.

Yeah, it seems the <style> and the <body> elements are in some conflict. So, I’m moving on to check that.

Styling and fonts

The <style> element section checks the support for template’s styling. I’m still using the same example, and as you can see below, some clients partially support the element, it appears buggy, and it’s not supported across all the email client versions.

Again, clicking on the number next to “Found on lines” takes me to the template and clearly shows what’s wrong. The best thing is that if a line gets declared as “Buggy”, there’s a quick explanation why it’s buggy and how to fix it, which may cut template debugging in half.

As for the fonts, one of the key parameters Mailtrap checks is the font-weight. Of course, this is assuming you’re using a standard font family.

Like before, the notes show you that the font-weight is partially supported, plus there’s a quick explanation of why it’s like that. But the check doesn’t stop there, you can also inspect the class selector, max-width, background-color, width property, max-width, and any other CSS or HTML element that might appear in the email.

I know, it may seem that there’s something seriously wrong with the template in the example, but it’s not. As you can see from the first screenshot, it has 93.8% support across different email clients. So it’s a good template, but there are options to make it even better.

When you’re done checking the template, it’s time to see whether it has a chance to trigger spam bots.

SPAM score

The spam testing results are under the Spam Analysis and they give you the overall score for an email. In the background, we’re running an up-to-date SpamAssassin filter to determine the score. And you can see which elements of your template drive the score up.

At the same time, under Blacklists Report, you’ll see if your domain and IP address appear in any of the most common blacklists.

Also, pointers to header information or buggy headers may also appear under the HTML Check tab, and the Spam Analysis tabs. For example, the Spam Analysis flags ‘Content-Type’ without the proper MIME header (check the screenshot below).

Automation in Email Testing (via Mailtrap API)

Yes, you can automate email testing and if you’re sending a lot of emails, you should consider doing it. Mailtrap allows you to automate via our RESTful API (available on the free pricing plan).

Sure, it requires some coding but you can let the API run the dull tasks and notify you if there’s a problem.

Since I mentioned Mailtrap HTML Check a few times, here’s an exemplary request script for the HTML analysis using PHP/cURL.

<?php

$curl = curl_init();

curl_setopt_array($curl, [
CURLOPT_URL => "https://mailtrap.io/api/accounts/account_id/inboxes/inbox_id/messages/message_id/analyze",
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => "",
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 30,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => "GET",
CURLOPT_HTTPHEADER => [
"Accept: application/json",
"Api-Token: 123"
],
]);

$response = curl_exec($curl);
$err = curl_error($curl);

curl_close($curl);

if ($err) {
echo "cURL Error #:" . $err;
} else {
echo $response;
}

And here’s an exemplary response to the request.

{
"report": {
"status": "success",
"errors": [
{
"error_line": 15,
"rule_name": "style",
"email_clients": {
"desktop": [
"Notes 6 / 7"
],
"mobile": [
"Gmail"
]
}
},
{
"error_line": 7,
"rule_name": "display",
"email_clients": {
"desktop": [
"AOL Desktop",
"IBM Notes 9",
"Outlook 2000-03",
"Outlook 2007-16",
"Outlook Express",
"Postbox",
"Windows 10 Mail",
"Windows Live Mail"
],
"mobile": [
"Android 4.2.2 Mail",
"Android 4.4.4 Mail",
"BlackBerry",
"Gmail Android app IMAP",
"Google Inbox Android app",
"Windows Phone 8 Mail",
"Yahoo! Mail Android app",
"Yahoo! Mail iOS app"
],
"web": [
"Yahoo! Mail"
]
}
}
]
}
}

The response you’re looking for is 200. If there are any issues, the system will return 401, 404, or 404 error messages.

Technical testing (outside the sandbox environment)

We’re now moving out of the realm of sandbox tests and I’ll cover some production tests or pre-production tests. These should be done either before sending an email campaign or after sending a batch of emails to a test cohort of recipients.

Thank you for exploring a snippet from Mailtrap's latest blog post on "How to Test Email." For a more in-depth guide on testing strategies, tools, and best practices, check outthe full article. Thank you for delving into the world of email testing with Mailtrap!