Close (X)

Blog

Email Marketing, Business & Monkeys

A/B Test Your Email Design and Content in MailChimp

November 4th, 2008 | by Ben

Did you know you can use some nifty new advanced merge tags inside of MailChimp to test different designs or content in your email campaigns?

It’s a great way to test two different:

  • Incentives, discounts, compelling offers (”Free Shipping” vs. “15% off your order”)
  • Headline font sizes, colors (do bright colors help?)
  • Headline word counts (long headline vs. short headlines)
  • Product images (standalone image vs. product in use)

to see which version gets more clicks and sales from your subscribers.

Plus, it’s super easy (no rocket scientists required). Here’s how to do it…

Just create an A/B campaign like you normally would (here’s a quick demo movie). You might want to set MailChimp to not automagically send a campaign to the winner. Set it to 50/50 test groups, or set it to “decide manually.” That way, you can look at your results and decide which campaign won yourself.

Then, in your email message, place this code wherever you want to test two blocks of content or designs against each other:

*|GROUP:A|*
This block of content/graphics will only show for group A recipients
*|END:GROUP|*

*|GROUP:B|*
This block of content/graphics will only show for group B recipients
*|END:GROUP|*

MailChimp will dynamically replace those merge tags with the proper content for each test group.

Members of test group A will see whatever you stick inside the *|GROUP:A|* tags, and members of group B will see whatever you stick inside the *|GROUP:B|* tag. Note that A and B groups are randomly and automatically chosen by MailChimp. That’s what’s so cool about our A/B tool.

Then, you can look at your campaign’s stats to see which content won.

Cool, huh?

And you can combine this with an A/B test of two different subject lines. Whoah. That’s like those fancy-schmancy multivariate tests the marketing nerds are always talking about.

For example, I sent an A/B test that showed group A got more opens, so that subject line won. But group B got more clicks, so that product graphic won. Next campaign, I’ll send it with subject line A, and product graphic B.

You can stick whatever you want inside those tags.

Text, links, pictures, tables. Even entire email template designs.

Yeah, you heard me right.

If you’re an advanced coder, you can switch to our advanced editor (where you copy-paste your own template code) and setup an A/B test of two totally different email designs. You’d do it like this:


<HTML>
<BODY>

*|GROUP:A|*
THE CODE FOR EMAIL TEMPLATE DESIGN A
*|END:GROUP|*

*|GROUP:B|*
THE CODE FOR EMAIL TEMPLATE DESIGN B
*|END:GROUP|*


</BODY>
</HTML>

Note how I took the HTML and BODY tags out of each of my different email templates, and then applied them around BOTH my A and B designs. I don’t think MailChimp cares about that so much, but I found it made things less confusing for MailChimp when you do a pop-up preview.

Give it a shot! But make sure you do plenty of testing before you send your campaign.

Click our “Send Test” button:

and send separate versions of A and B to yourself plenty of times to make sure you’ve got everything setup correctly. Seriously. You have to test your A/B tests, too.

Spread the monkey love:
  • TwitThis
  • Digg
  • Facebook
  • del.icio.us
  • Reddit
  • StumbleUpon
  • description
  • Google
  • LinkedIn
  • Ma.gnolia
  • MisterWong
  • Netvouz
  • NewsVine
  • Slashdot
  • Technorati
  • YahooMyWeb
  • BlinkList
  • Design Float
  • Mixx
  • Pownce
  • Propeller
  • Webnews.de

5 Comments

    • Kassy says:

      Oh this is so helpful and exciting!!! (now I just have to wait for the ability to customize content based on segments)

      Thank you!

      PS did you send out an email on this?

    • Angela Walker says:

      When you set this up for just content, what do you choose for the A/B set-up? Do you choose any of the three variables (subject line, from name, time of day), but keep the same information in each one?

      • Ben says:

        Yes, you’ve got the right idea. I’d do delivery times, and make the differences negligible. Or, test multiple things at once (multivariate testing).

    • Angela Walker says:

      Oops – on previous comment meant to check “Notify me by email” for return answer. Possible to do on your end?

Leave a Reply

* indicated required
http://www.mailchimp.com/nonrestrictiveocean.php