Now, they’re shifting their attention to measuring “engagement.” They defined engagement as opens, clicks, and having an email moved out of the spam folder. This is similar to Gmail’s approach to leaving images on if the recipient knows the sender.

How Does This Change Things?

Hmm. If ISPs are starting to look at how engaged your subscribers are, how could an email sender use this to their advantage (beyond simple list  segmentation)? Perhaps you could send email a little differently through your delivery servers, based on your subscribers’ engagement activity? For example, if you knew half the people on your list were active users, but the other half not so much, wouldn’t it be smart to deliver the campaign to the engaged people first, then the others last? It would really suck to only get a small portion of your list delivered before an ISP decided you have poor list management practices, and blocked the remainder of your message.

Yes, MailChimp does all that. Automatically, and behind the scenes. That’s the reason we launched the List Activity Score back in March. We rank every single user on your list by their engagement, then we prioritize email delivery through our network based on overall list activity score. One of the many ways our nerds in the lab keep striving to improve email marketing.

Learn more about our List Activity Score

Related:
How sending to old lists will kill your deliverability

To be honest, I’ve never seen that warning, and have no idea what exactly triggered it. As you can see, the email was also sent straight to gmail’s junk folder.

On the surface, nothing about the campaign looks bad. The general content of the campaign is fine. The sender is not in a risky business (it’s a church). Their email delivery infrastructure (ahem, mailchimp) is fine. So what gives?

We ran the campaign through our inbox inspector, and got the following “spamminess” score:

Notice it failed Barracuda, Cloudmark, and Postini. It also triggered one rule in Spam Assassin (which, btw, is used in some way, shape, or form by just about all the other spam filters) that got 2 whole points. By now, we should all know how spam filters generally work, and that you shouldn’t use “trigger words” like “FREE!” or “BUY NOW!!!” in your content. But even when you do, those words usually only get assigned a few fractions of a point. Go to this list of spam assassin rules and CTRL+F for the word “FREE!” to see what I mean.

But when you see something getting 2 whole Spam Assassin points like this, something’s very wrong.

The rule that was triggered? The message contained a URL listed in the URIBL Blacklist. Upon closer inspection, it turns out they were using a URL shortener (you know, something like tinyurl.com). I’m not going to name names, but this URL shortener wasn’t quite as well known as most of the others I’ve heard of. No idea if it has a bad reputation, but if it’s new on the scene, chances are high that it doesn’t have enough of a reputation.

In general, URL shorteners are great tools that serve a good purpose, but spammers have abused the heck out of them to disguise their (already blacklisted) links.

In response, some spam filters make a habit out of “clicking” all URLs in an email, just to follow redirects from URL shorteners, and analyze the landing page they’d take you to. Which, btw, can lead to some unintentional unsubscribes, but that’s another topic.

If this is all new and fascinating to you, check out this article from Laura Atkins at Word To The Wise: Failed Delivery of Permission Based Email. She covers a few other seemingly innocent but oft-abused URLs that can get your messages blocked.

But it’s not just URL shorteners at risk. Any domain name with a bad reputation can get blocked. For example, there’s this article from yours truly:

Is Your Domain Name Getting You Blocked?

Finally, if you’re a MailChimp customer be sure to check out our built-in, one-click email checker: Inbox Inspector. It can help you prevent renderability and deliverability problems before you send your campaigns.

Matt Vernhout from EmailKarma reports that Gmail is now turning images ON by default, so long as the recpient has sent YOU, the sender, two messages in the past (kind of a neat way to make sure there’s a trusted relationship). Here’s the post from the official Gmail Blog.

There’s another catch — your emails to the recipient have to be authenticated (SPF or DKIM). As a reminder,  Authentication is a method used by many ISPs to judge whether or not an email is trustworthy (learn more at the Online Trust Alliance’s website). All major forms of authentication are built-in and automatically turned on for all your MailChimp campaigns.

As Matt points out, it’s almost worth it to get rid of any “DO-NOT-REPLY” statements you might be using, and actually encourage your recipients to send you emails. If it sounds a little too scary to add a “send us feedback!” link for your entire list, just add that for Gmail subscribers.

Here’s how you can segment your list and send only to your subscribers @gmail.

Seriously, I had to wash my fingers after typing such nasty stuff, and I couldn’t look at myself in the mirror for a day or two.

In addition to the yuckiness, I typed in a bunch of stuff about gambling, and some phishing type content. And I made sure to use all caps, with lots of exclamation points (see why spam filters hate that).

Then I ran it through our Inbox Inspector’s Spam Checker tool…

Turns out I passed all the major spam filters!

My Spam Assassin score wasn’t great (anything over a 5 is DOA but I like to stay well below 3). But that seems to be because of other problems.

In terms of the extremely disgusting alt-text descriptions I used, they don’t seem to have triggered anything at all:

Got any questions you want the MailChimp team to answer here on the blog? Submit them here.

Learn more about MailChimp’s Inbox Inspector:

Some highlights from the article, plus links to related email marketing resources from MailChimp, are below.

What’s the ROI for email marketing?

According to the Direct Marketing Association, email marketing is projected to return $43.52 for every dollar spent in 2009.
See also:

Tracking ROI from your email marketing with MailChimp

Case Study: $157,000 from one email campaign

Google Analytics Blog: Tips for tracking email marketing campaigns

How much email gets lost?

“A very small percentage of emails the ISPs [Internet service providers] handle is legitimate—90 percent of it is spam,” says David Atlas, senior vice president of worldwide marketing and sales for certified email provider Goodmail. Of the ones that are legitimate, email deliverability vendor Pivotal Veracity reports that roughly 20 percent don’t even get delivered.

See also:

Deliverability on the MailChimp blog

Check your email campaigns in major spam filters with one click

MailChimp’s List Activity scores launching soon

What’s the definition of spam?

“Consumers today see less spam than they’ve seen in the past,” MAAWG’s Stiles says, “[but] there’s actually more spam than there’s ever been.” But even if you’re not selling Viagra, spam is in the eye of the beholder. Some experts define it as anything a recipient doesn’t want; if the request was for emails to be sent monthly and you send them daily—you’re spam. “Companies will say, ‘But I’m CAN-SPAM compliant,’” Stiles notes. “[That] means you won’t wear an orange jumpsuit and handcuffs. It doesn’t mean you have the right to send email.”
See also: How legit email marketers can prevent spam complaints

Is authentication important?

David Daniels, a Forrester vice president and principal analyst, anticipates that by the second quarter of 2009, if a sender’s address is not authenticated, it won’t just be more-carefully screened—it will be immediately dropped.
See also:

Authentication is related to deliverability

MailChimp email authentication guide (see which ISPs check for which forms of authentication)

Will Yahoo block messages that aren’t authenticated?

More info on MailChimp’s Inbox Inspector here or just watch this movie to see how it all works:

Each list that you manage in MailChimp will have its own “List Activity Rating.” Below is a mockup of what it’ll look like:

Told you we’re suckers for stars.

Your list activity score is based on an algorithm that tells us how “active” and “engaged” the members of  your list are. Generally speaking, the more active your list is, the better your deliverability (here’s what happens if you send to a list that’s not active). And since our job as an ESP is to maximize your deliverability, we need to know whose lists are active, and whose lists are not. Your list activitiy score will determine how we deliver your emails from our system.

Some background information

For the last 7 months or so, we’ve been doing some secret, “behind the scenes” research by scanning all campaigns sent from our servers.

We’ve been experimenting with the concept of “user activity.”

Basically, what if we could tell how “active” or “inactive” your subscribers are? And how can we use that to determine how “active” your entire list is?

Armed with that knowledge, how could we improve the overall deliverability of our system?

For example, if you send an email campaign, and one of your subscribers has never, ever clicked or even opened any of your previous campaigns, that’s not a very active user. And what if a big chunk of your list is composed of similar “inactive” members? That’s probably an indication that you’ve got a bad/old list mixed in there. We’re not saying you’ve done something wrong, or that you’re a spammer. But your list is old. We won’t punish you for that (unless our abuse team thinks you’re breaking our terms of use). But knowing you have an old list, should we allow you to send to that list across the same IP addresses used by other MailChimp customers, who have extremely active/clean lists?

How we determine “list activity”

It’s not just about opens and clicks. We look at the number of bounces across your list, the number of spam complaints, and the number of unsubscribes.

Your list activity is an overall score based on the “activity” of each member in your list. Here’s a gross over-simplification of how it works (we’ll post more details later).

Let’s say John is one of your subscribers. John starts with a certain neutral score. John opens an email. John gets a +1. John clicks a link in your email. +2. You send a few campaigns, and John soft bounces. -1. Next campaign, John opens. +1. Mothers Day rolls around, and your boss tells you that “we really need to blast out something totally irrelevant to our list, or we won’t meet our quota.” You have no spine, and you agree to send John something he didn’t give you permission to send him. John clicks the spam button. He gets a -2 and gets removed from your list.

Now imagine that measurement happening across your entire list of members. Average them all up.

That, roughly speaking, is your overall list activity score.

Double opt-in is not perfect.

In the past, we separated our IP ranges based on whether or not a customer used double opt-in lists, vs. single opt-in lists. The problem with that is the sheer number of single opt-in lists dwarfs the number of double opt-in lists. Also, those who use double opt-in lists tend to be occasional senders, not high-strung, “gotta have bazillions of subscribers and blast an email out every week” kind of people. I’m an occasional sender, which is why I don’t care one bit if a certain percentage of signups don’t complete the double opt-in confirmation process. Hey, the must not have wanted it that bad. Nothing wrong with that. Plus, I just prefer to send emails when I have useful stuff to say. Which is why I usually send quarterly(ish).  The problem, if you pool a bunch of us into a range of IPs together, is that very little email volume goes out. No email volume means no history. No history means no reputation (here’s an article we posted on the reputation of new IPs). If you get a sudden spike in volume, the lack of reputation can actually hurt deliverability.

Human reviewers are not perfect.

We also have a human review team here at MailChimp (I’m part of it too, sometimes). We’ve developed a “sixth sense” when it comes to reviewing new signups. We can look at their contact information, and in the blink of an eye, determine whether or not they’ll be a) a good user, b) an ignorant and problematic user, or c) kicked out the door. At some point, I documented something like 25 different criteria I use to judge someone. For example, if you sign up for a free trial with MailChimp, and you used ALL CAPS in every single field of our signup form, that’s not a good sign. If you don’t have a favicon on your website, you’re not a details kinda guy. If you don’t have a website at all, something’s fishy.

The human review team also had the power to “promote” certain users to a “trusted” IP range, away from the big shared pool. This decision would be based on whether or not the sender had a good history with us. The problem with that approach is you’re basing things on the customer.

Not the way they behave.

And customers change. Especially in desperate economic times. We saw that in the dot-com fallout days, and we’re seeing hints of it now.

I’ve approved some great customers who later did some stupid things. One customer who works for a very large corporation, with a near-perfect email history, recently volunteered for a local non-profit and imported their stale 4-year old list. It got one of our IPs blacklisted. Good user. Stupid behavior. Our abuse team can get our IP delisted, and restore our reputation relatively easy. His reputation? Maybe not so easy. And now, we will change the way we deliver his emails in the future, until he builds his score back up.

Moving forward, the system will judge lists based on their behavior, not by the reputation of their owners.

Stellar list scores = stellar deliverability

We’ve been running this for 7 months behind the scenes, and so far it’s been great. We’ve been comparing our list activity scores to overall deliverability (using ReturnPath’s Mailbox Monitor), and it’s spot-on.

Those with stellar list scores get stellar deliverability.

Every list on our system now has a list activity score, and in late March they’ll be revealed in your account.

Again, don’t think of this as a way to take people with “bad emails” and throw them into some swampy black hole of IPs. That would hurt MailChimp as much as it would you.

This system is not meant to punish anybody (our abuse team is working on that system. seriously). This is meant to show you how relevant your email marketing is, and how clean your list is. No matter what your score is, MailChimp’s deliverability will be great. But if you’re sending relevant email campagns to permission-based email lists, your deliverability will be better than great.

Think of this as an incentive. To do good.

MailChimp’s abuse desk runs Cloudmark to perform occasional “customer audits.” We basically scan for problem campaigns on our system that might jeopardize the deliverability of our servers. What’s Cloudmark, why do we use it, and how does it work?

Cloudmark is an advanced “message security” system that protects more than 300 million inboxes and works with more than 100 of the world’s largest ISPs and mobile operator networks such as EarthLink, Comcast, Cablevision, Charter Communications, Cox Communications, NTT Communications, Sprint Nextel, Virgin Media and Swisscom, as well as hosted messaging providers, including domainFACTORY and NuVox.

So if you send lots of email marketing, it’s kind of important to know who they are.

But how does their spam filtering technology (its fingerprinting algorithm) work?

Well, it’s a secret. Understandably so, because if they told everyone how they work, that would kind of defeat the purpose.

But here’s what they will tell you (from their website sales material):

Cloudmark’s Advanced Message Fingerprinting™ algorithms were designed to target sophisticated spamming and virus proliferation techniques. Unlike rules, Cloudmark fingerprinting algorithms are extremely lightweight, each optimized to perform only minimal processing on a message. As a result, message throughput is extremely fast and less processing CPU is required.

And here’s how they explain their Fingerprinting algorithm:

So they’re taking chunks of your message (which I assume could be content, senderscore reputation, and code), and taking it out of the context of your email campaign. I don’t know if this is done for speed, or as some kind of “double blind” methodology or what. Then they classify the chunks into “fingerprints.” Then, they compare those fingerprints from your campaign with other fingerprints in their database that have been classified as spam.

This is where I invite any geek out there who knows way better than me to please comment below. Please.

What to do if Cloudmark blocks you

If you get blocked by Cloudmark (and our abuse desk sent you to this page), our recommendation is to take a long, hard look at your content. There’s something in there that looks spammy. Given that Cloudmark is installed across 300 million inboxes and +100 ISPs around the world, it’s safe to say that your campaign looks spammy to a LOT of people.

If you’re not sure what “looks spammy” means, I’m not so sure you’re ready to be sending lots of email marketing.

Okay, maybe that was a bit out of line. I work at the abuse desk, so I get jaded sometimes. So here are a couple resources you need to read quick:

• Most common spam filters triggered by MailChimp users
• How spam filters think, and how to avoid them

If you’re looking for a simple, silver bullet kind of answer for “how to just get me past the spam filters” prepare to be frustrated. There is no single answer. The best answer I’ve been able to tell people is:

1. Open up your email program’s junk folder.
2. Look at what spammers do.
3. Then, don’t do that.

Cloudmark is everywhere

We’re members of the ESPC, and once sat in on a presentation that Cloudmark gave to the group. It was fascinating. Mostly because it was a “marketing guy” talking, who actually knew his stuff. No offense to marketing guys or anything. He knew about this stuff, and in the cases where he didn’t, he was smart enough to admit it. I distinctly remember a slide in his presentation where he showed almost every single major ISP in North America using Cloudmark. IIRC, the only ISP not on the list was AOL. They’re even partnered with ReturnPath (who we’re also partnered with) so that they can pull in sender reputation data.

If you run an ESP (and manage the abuse desk at an ESP), it’s the kind of slide that makes you gulp really loud. So I’m really glad we’ve got this in place for our abuse desk. I’ll post something later about how we’re using it to make better decisions about email abuse, who we warn, who we suspend, and who we shut down.

The backstory is a MailChimp customer sent a campaign to an email list that they collected at an event a long, long time ago (Related: How old lists will kill your deliverability). One of their recipients forgot who the @#%& the sender was, and reported the email to SpamCop.

Forget about the whole issue of whether or not the sender is an innocent victim here, because their list was ‘opt-in.’

What really matters is the sender’s domain name could be tainted, and all their emails (no matter where they send from) could be blocked all over the globe.

Here’s how that happens.

See the “spamvertized web site” links in the screenshot?

Those are some of the domains that SpamCop found in the reported email.

The 3 domains that you see in the screenshot above belong to MailChimp.

The domains listed below them (that you can’t see) are domains that belong to the sender of the email campaign (I’m protecting their privacy here).

There are 3 ways we can get our domains de-listed from SpamCop:

1. Shut down the sender (the fastest way to get delisted)
2. Respond to this report, and provide documentation that proves the sender obtained opt-in permission from the recipient, so “as you can see, this is all probably a simple misunderstanding.”
3. It behooves me not to tell you the third way.

One way or another, MailChimp’s Abuse Desk will get our domains delisted from SpamCop. But if we find out that someone has intentionally violated our terms of use, how hard do you think we’ll try to get the sender’s domain names delisted?

If we find out the sender purchased an email list, or they had an old email list and thought MailChimp would be a convenient way to “clean it,” we’re not exactly going to go out of our way to help their domains get delisted from SpamCop as we show them out the door.

The point I’m trying to make is that anti-spam systems “remember” domain names that they find inside of reported spam.

So if we end up deciding to shut down this MailChimp customer with extreme prejudice, and they move to some other email service provider (ESP), their domain will still be remembered as an abuser by SpamCop (and probably other email gateways and firewalls around the globe too).

If you have bad email management practices, you can run, but you can’t hide from your own email reputation.

How do you prevent this from happening to your company’s reputation?

• Never send unwanted email
• Don’t surprise anybody with emails they wouldn’t expect
• Don’t assume that people on your list remember who you are
• Don’t send to old email addresses
• Collect proof of opt-in, just in case you’re reported to SpamCop. Without it, ESPs have little recourse but to shut down your account.
• In your emails, always include some kind of reminder as to how you got the recipient’s email address (you’re receiving this email because…”). Bare minimum, put that in your footer. If it’s your first email campaign, consider making it your first paragraph.

• Forgetting to include physical mailing address in the email footer
• Failure to properly clean unsubscribers from a list
• Misuse of the word “free” in subject lines
• Not warning about inappropriate content in your subject lines

The documents are located under the “MailChimp Experts” group in the MailChimp Jungle community: Download them here.

The content is nothing new, as links to these lawsuits are all over the interwebs. But we just wanted to put it in a handy, printable guide. I’ve also included some little tips on “how this could happen to you” plus some food for thought at the end.

It makes a great gift for clueless clients, stubborn managers, and over aggressive sales people. Or, just print out a copy and drop it on your company lawyer’s desk.

Update: Rob Hassett, from InternetLegal.com, has contributed a couple topics on “Non-profits are not exempt” and “Affirmatve consent does not create an exemption from CAN-SPAM.”

While you’re perusing the MailChimp Jungle, be sure to check out Isabel’s collaboration for an email marketing project contract, and Tanya’s Yahoo Shortcuts discovery.