Close (X)

Blog

Email Marketing, Business & Monkeys

MailChimp’s A/B Email Tester Can Be Really Mean

August 11th, 2008 | by Ben

A lot of people tell us they appreciate the compliments that MailChimp gives them when they do their email marketing. Our chimp chatter feeds and little love notes seem to put a smile on our customers’ faces.

But this weekend, I discovered there’s a mean side of MailChimp, too…

I ran an A/B test on a recent campaign of mine, and I actually thought my Group B subject line would win (I thought more people would open a subject line that hinted at privacy policy changes, rather than new features). The A/B split proved my theory wrong. My results:
Group A Subject Line (31% open rate): MailChimp v3.2: RSS-to-email, Batch Unsubscribe, 16.5 languages…
Group B Subject Line (27% open rate): MailChimp v3.2: Privacy policy changes,  RSS-to-email, Batch Unsubscribe, 16.5 languages…

So I guess people aren’t as paranoid about their privacy as I am.  Good thing I did an A/B test, because the open rate for my crappy subject line would have meant almost 2,000 people wouldn’t have opened! If you’re an eRetailer, 2,000 lost opens can mean a lot of money.

Anyway, just to pour salt on my wound, MailChimp added a little commentary to my Group B attempt:

ab-split-loser.jpg

Aaah, nothing like being called a loser first thing in the morning. This is gonna be a good day.

Speaking of A/B testing, I hear that FutureNow’s book on A/B testing, “Always Be Testing” is now shipping.

Spread the monkey love:
  • TwitThis
  • Digg
  • Facebook
  • del.icio.us
  • Reddit
  • StumbleUpon
  • description
  • Google
  • LinkedIn
  • Ma.gnolia
  • MisterWong
  • Netvouz
  • NewsVine
  • Slashdot
  • Technorati
  • YahooMyWeb
  • BlinkList
  • Design Float
  • Mixx
  • Pownce
  • Propeller
  • Webnews.de

One Comment

Leave a Reply

* indicated required
http://www.mailchimp.com/nonrestrictiveocean.php