We analyzed 1,720 A/B tested campaigns, sent to 6,226,331 inboxes (we only included campaigns sent to over 500 recipients).

Overall, we found that A/B split winners did better than non-tested campaigns. By how much?

Before we give you the exact numbers, test your email marketing aptitude and pick the winning subject line from these real life A/B email tests conducted by our customers:

Think you know which ones were the winners? They’re all the subject lines in the first (A) column. In general, we found that:

• Subject lines with company names in them did better, and they did better when the company name was near the beginning of the subject line (perhaps it was the increased recognition factor)
• Shorter subject lines seemed to work better than long subject lines (the difference in open rate was more noticeable when they differed by 30 characters)

How Much Better Are A/B Tested Email Campaigns?

We compared the email marketing stats of A/B testers against the average email marketing stats across all MailChimp customers.

Average open rate across all lists: 21.7%
Average open rate for winning A/B groups: 24.1%

Average click rate across all the lists: 4.7%
Average click rate for winning A/B groups: 5.5%

*We analyzed 1,720 A/B tested campaigns, sent to 6,226,331 inboxes (we filtered down to emails sent to over 500 recipients).

As you can see, those who take the time to A/B split test their email campaigns get better overall results than those who don’t. For some extremely large lists, those few percentage points can make a huge difference in eyeballs, clicks and conversions.

So right now, I’m looking at A/B split data across tons of campaigns. This is a real gold mine for email marketing research, so I was hoping to discover some universal truths that debunk what the “experts” tell us. Like, “Wednesdays are always best, not Mondays, so in your face!” Turns out, the only thing I’ve really learned is that all the expert tips out there on email marketing (mine included) are worthless. The results are all over the board. At first I was frustrated by the lack of consistency in all this data. Then, I realized that this is exactly the whole point—there is no silver bullet. Everything depends on your company, your offer, your list, etc. The only way to tell what works is to test, test, test.

But I am looking across all these amazing stats, and noticing some interesting patterns. For example, personalizing your subject line (such as with “FNAME”) doesn’t seem to help open rates very much. In fact, it can actually hurt…

Here are two representative campaigns that A/B tested subject lines with FNAMEs in them (I’ve disguised company names with “Acme”):

Example 1:
Sample A: Acme Energy Challenge: December Newsletter (40% open rate)
Sample B: *|FNAME|*, Your December Challenge Newsletter (37% open rate)

Example 2:
Sample A: Acme newsletter: Limited Quantity Parts (56% open rate)
Sample B: *|FNAME|*: limited quantity parts at special savings (46% open rate)

I do realize this is just two measly examples, but they pretty much represent what I’m seeing across all the campaigns in my data set. One subject line, then the same thing again with “FNAME,” or “FNAME:” in front of it. In almost every single case, the subject line with FNAME included was beaten by the subject line w/out it.

So Does FNAME Hurt?

I don’t think “FNAME” makes people not open. I think the problem is that FNAME is a waste of space in your already-cramped subject line slot. It’s just a hunch.

In examples 1 & 2, the FNAME tag basically replaced the company name in the subject line. And as we all know, your company name can help make your subject line look a lot more reputable and relevant (see: Subject Line Comparison Study).
Consider this example, where the winning subject line actually did include the recipient’s FNAME:

Example 3:
Sample A: *|FNAME|*, Get a laugh out of taxes, fire pricey ink, and more! (7.85% open rate)
Sample B: Get a laugh out of taxes, fire pricey ink, and more!’ (6.84% open rate)

Notice the winning subject line didn’t really win by all that much (sample size for A and B was 43,000 recipients each). Also notice that unlike the two campaigns above, this campaign’s subject line wasn’t really hampered by the FNAME. It didn’t replace the company name. In fact, there was no company name at all in either subject line. Maybe that’s why the overall open rate was so low (see: Subject Line Comparison Study)?

IMHO, the real “hook” of this subject line was the “Get a laugh out of taxes…” Everybody wants to laugh, right? And jamming an FNAME in front of that doesn’t really push that hook out of view, or replace it altogether. Could even be the case that using FNAME does help boost open rates for “informal” subject lines. But I have no idea. I guess you just have to run your own A/B tests to find out.