Click to see the winner

When I created this email newsletter, I had two different subject line ideas:

A) MonkeyWrench – Autoresponders, Turks, Freshbooks, and Webhooks

B) Turks, Freshbooks, autoresponders, and webhooks

Quick – Can you  guess which subject line got more opens? Turns out it got more clicks and fewer bounces, too!

Luckily, there’s no need to guess with MailChimp’s patent-pending built-in A/B testing tool. It’s totally automatic. I’ll show you how it works…

Log in to MailChimp, and click on the big orange “Create Campaign” button. Choose the “A/B split” option:

Next, you’ll be asked to setup your A/B test:

the setup page is pretty straightforward, but let’s break it down into pieces.

First, tell MailChimp what you want to test. I’m testing 2 subject lines:

Next, tell MailChimp how many subscribers you want to test on. Pick a random slice of your list that’s enough to make you feel confident about the results. I usually pick 20% of my list, but that might be different based on your list size.

You just drag the little slider left and right:

In the scenario above, MailChimp will randomly select 10% of my list, and send subject line A, and another random 10% will get subject line B.

Next, tell MailChimp how it should choose the winner:

Above, I’m telling MailChimp to “Wait 6 hours, then look at open rates. Whichever subject line got more opens, use that for the remaining 80% of my list.” You can make it wait only 1 hour, or stretch it out to several days if you want. I think 6 hours is long enough. Plus, I’m kind of impatient. If you like complete control over everything, choose “manually” and MailChimp won’t automatically send anything. You can log in and choose the winner. But where’s the fun in that?

Anyway, next I just go about creating my email campaign like I normally do, except when it’s time to enter my subject line, I get two boxes:

Plug in my two subject line ideas, and I’m done.

I then scheduled the campaign to send at 9am ET on Monday morning.

Then, I just sit back and wait. MailChimp does all the rest for me.

No need to generate random slices of my list, then generate more random slices, then generate the remainder of my list, and no need to build 2 separate campaigns, then a 3rd winning campaign. It’s just automatic. It’s so easy, you can run A/B tests on every campaign you send. Always be testing!

It saves me a ton of time, so that I can do other stuff.

Like blog about what I just did.

Also See: Those who A/B test get 10% better open rates