Close (X)

Blog

Email Marketing, Business & Monkeys

Real stats: How sending to old lists will kill your deliverability

November 21st, 2008 | by Ben

We have a customer with a relatively large list of about 311,000 opt-in subscribers. They’ve been collecting opt-ins from their site for years now.

About 240,000 of them are “old” (inactive) subscribers. About 70,000 are relatively “new” (active) subscribers.

They recently segmented their list and sent the same newsletter to each group (separately) over the same IP address, about 6 hrs apart from each other. Around 2pm, they sent the newsletter to the large, inactive list. Around 8pm, they sent the same newsletter to the active list.

The results are eye-opening…

Stats for the “Inactives” list (241,832 recipients):

Spam Complaints: 43
Open Rate: 6%
Click Rate: 2.4% (and 7,688 total clicks)
Unsubscribes: 264
Bounces: 6,878 (2.8%)

Stats for the “Actives” list (69,642 recipients):

Spam Complaints: 3
Open Rate: 36.3%
Click Rate: 7.4% (and 6,925 total clicks)
Unsubscribes: 96
Bounces: 128 (0.18%)

The number of bounces for the “old-members-list” is what I’d call very high. The number of abuse reports (43) is what I’d call disturbing. These are humans clicking the “this is spam” button for a run-of-the-mill company newsletter.

More interesting is what happened to their deliverability for the old list. We measured it using ReturnPath’s Mailbox Monitor service. Here’s a screenshot:

As you can see, their emails went 100% missing from AOL and Hotmail, 80% from Yahoo, and 70% for Comcast.

Also, it’s almost a day later, and 20% of the “sent-to-old-members” list still hasn’t been delivered to Yahoo, because they keep getting deferred.

My guess is the sheer number of spam reports from the old list is what got them blocked, and therefore their emails have gone missing (here’s some explanation from AOL on how they measure IP reputation).

If you’ve got a bad reputation, you’ll get blocked by AOL (and most of the major ISPs). Comcast’s Postmaster lists how emails get throttled based on your “Sender Score.” The lower your score, the slower you’ll get delivered.

So how is your score determined?

AOL measures your IP reputation by:

  • Spam complaints (from humans clicking the “spam button”)
  • Spam filter triggers
  • Bounces from undeliverable addresss (i.e. old lists)

I wish we had deliverability stats to show you how the “active-members-only” went, but we forgot to track the good one (d’oh). Maybe next time.

Even without a side-by-side deliverability comparison, the end result is clear: old lists suck.

They generate too many abuse complaints, which can get your entire campaign blocked by ISPs. Then, the active members who want your emails never get your message.

Related Stories:

Ever thought about segmenting your email campaigns so that they only go to the “active” members on your list? Here’s how to do that in MailChimp.

Spread the monkey love:
  • TwitThis
  • Digg
  • Facebook
  • del.icio.us
  • Reddit
  • StumbleUpon
  • description
  • Google
  • LinkedIn
  • Ma.gnolia
  • MisterWong
  • Netvouz
  • NewsVine
  • Slashdot
  • Technorati
  • YahooMyWeb
  • BlinkList
  • Design Float
  • Mixx
  • Pownce
  • Propeller
  • Webnews.de

Tags: , , , , ,
Posted in Deliverability, Tips, Tricks, Best Practices | 10 Comments »

10 Comments

    • Daniel Mills says:

      Very Interesting, fit’s my experiences as well. What do you consider an “old” list?

      Reply to this comment
    • Ben says:

      @Daniel – At MailChimp, we have a big list of users. According to our terms of use, we can contact all of them, since they’re registered users. But most of the time, whenever we send a “system alert” out to all registered users (such as about planned server maintenance), we segment the list to “users who have logged in within the past 6 months.”

      We figure if you haven’t logged in for 6 months, you’re not very active, and you could care less about temporary server downtime.

      There have been times when we’ve sent to *all* members of the list, such as for extremely critical system alerts that affect accounts.

      Those campaigns always generate excessive abuse complaints, bounces, and angry emails (from our own users) Those are not good days for me.

      Sometimes, you just *have* to contact all your customers (even old ones). But you have to understand the repercussions on your deliverability. You WILL lose a lot of members, ISPs WILL defer or block your emails, and you WILL get reported for spam.

      If you have to contact those customers, permission (and *proof* of that permission, such as opt-in IP, date, etc) is critical.

      If you send a massive campaign to lots of inactive members, and you get blocked by ISPs, or you end up on a blacklist, your ESP (email service provider) may be forced to shut your account down and will require that you show proof of opt-in in order to be reinstated.

      Reply to this comment
    • Daniel Mills says:

      @Ben
      yeah, we’ve experienced that as well and we know about even worse results than in your example. There are companies who have collected email addresses for years (like online shops etc.) but have not used the addresses for marketing. When they finally start using their list for the first time (w/o segmenting), they get bounce rates in the 30%s and spam complaints that scare the hell out of me.

      And what I found is kind of common for more and more email receipients is to hit the “spam button” when they are tired of email messages, regardless if they double-opted-in just days ago and there’s a prominent unsubscribe link.
      I compare that to the behaviour of typing URLs into the google search bar instead of the address bar of the browser, it’s just more convenient for the user and for him it has the same effect: he does not get your messages anymore. But as an email marketer – frightening.

      Reply to this comment
    • Kevin says:

      Another reason old lists suck is that it’s more difficult for a newer person in the organization to fully understand where all those addresses came from.

      Since it’s impossible to say to your new employer that you’ll only email folks that opt in after you were hired, you are left with a long list of people that may just have wanted to win a free sub sandwich from your fishbowl drawing in 1994.

      In email marketing, old lists are almost always trouble in the making.

      Reply to this comment
    • e-marketing says:

      old lists are a pain. unless you have a hygiene process then it’s difficult to clear up.

      Reply to this comment
    • Tim Linden says:

      Well I just disabled our auto emails asking old members to come login! Will have to record our current levels and see how it affects future levels.

      Reply to this comment
    • DJ Waldow says:

      Ben –

      Well said. These are the types of posts that we need to get marketers (and those C-level folk) to read. However, I also agree 100% with Stephanie’s follow up – http://blog.deliverability.com/2008/12/risk-reward-take-ii-on-the-mailchimp-study.html. I noticed that you all had a nice “comment exchange.”

      I had a similar conversation on the Email Marketers Club this week. Just posted about it – http://blog.bronto.com/2008/12/03/how-do-i-clean-a-7-year-old-list-with-28-million-emails/

      Carry on!

      dj (Bronto)

      Reply to this comment

Leave a Reply

* indicated required
http://www.mailchimp.com/nonrestrictiveocean.php