Testing Tips from Conversion Rate Squirrel

March 26, 2008 – 10:27 am

So I’ve been toying with Google’s Web Optimizer tool lately, because I plan to do some A/B testing on the MailChimp website. I wish A/B testing websites was as easy as A/B testing email. Anyway, in my research, I stumbled upon the Conversion Rate Experts, who actually make A/B testing and conversion rates entertaining (okay, it’s because of their Conversion Rate Squirrel critter—I’m a sucker for cartoons).

If you haven’t already, I highly recommend you sign up for Conversion Rate Squirrel’s FREE “Life-Changing SuperConverters Newsletter”. Mainly because it’s full of useful tips that might help you with your own website. Also because it’s a fun, well-written newsletter that might give you ideas for your own email newsletter.

I asked the Conversion Rate Squirrel if he could offer up some tips to MailChimp customers for A/B testing their emails…

Here’s what the squirrel told me…

1. My top tip for any kind of split testing, is to make sure you have enough data to get a statistically significant result. Many of the multivariate & a/b split testing packages for testing websites have these already built-in… but too many people jump to conclusions before gathering enough data, and end-up basing a whole marketing campaign on some invalid results (I think that’s Tip 24 of our squirrel article).
2. In terms of the email content… I’m sure you guys already have a great understanding of ‘what to test’ (subjects, formatting, p.s’s etc…). Our approach is to spend loads of time learning about your websites visitors/customers - to understand their needs, and then make sure what you send them is what they want The results from our ‘Survey With A Difference’ email campaign, has steered our whole business and marketing plan
3. My final tip, we like to make sure what we send is entertaining - and always ask for feedback from our readers to make sure we’re not irritating anyone

A couple things got my attention.

• Interesting how they conducted a customer survey that “steered their whole business and marketing plan.” If you want to put together a survey of your own, be sure to check out our friends over at SurveyMonkey.
• They test content of their emails, which is a no-brainer. But one thing I don’t think I would have thought to test would be my “p.s’s” at the end of my messages. Interesting thought. I don’t even think I’ve ever written a P.S. in my newsletters, but if Conversion Rate Experts are doing it, I’m gonna try them too.