Close (X)

Blog

Email Marketing, Business & Monkeys

Ethnographic User Testing and Chicken Salad

December 12th, 2008 | by Ben

So I got a call from our accountant the other day. She told me she was having a few issues with MailChimp, and she wanted my help. I told her, “Stop everything you’re doing. We’ll buy you and your entire staff lunch if you’ll let us swing by your office and watch you build your email campaign.” She agreed, asked her team what they wanted to eat, and sent me the order.

I debated whether or not to just take them all out to a restaurant. Or to give them a big gift certificate, so they can just go themselves some other time.

But I thought it would be neat to watch them eat their lunch with one hand, and try to build an email campaign in MailChimp with the other. Aarron Walter, our user interface guru, calls this ethnographic user testing. Watching users “in the wild.” Dan, my co-founder, calls it “beer can usability.” Watch real people trying to use your app with a beer in one hand, slightly inebriated.

User testing “IRL” is hard to do, but you find out all kinds of weird stuff about your product when you do…

For example, here’s one thing it uncovered.

We all know that you should never, ever, ever, copy-paste content from a Microsoft Word document into MailChimp’s (or any) WYSIWYG, right? Because Microsoft adds so much junk to the underlying code? And it screws up everything?

No, we don’t all know that. People are constantly copy-pasting from Microsoft Word. We don’t blame them, because it’s natural. But it breaks every time (for the record, Microsoft calls all that junk “roundtripping code” so that your content and formatting can theoretically be re-purposed across Word, Excel, Powerpoint, etc.

Anyway, people copy-paste from Word all the time, and will never stop doing it. And when it breaks, they of course blame it on MailChimp. Especially if they’re in a hurry to get an email campaign sent. We tell them not to copy-paste from Microsoft Word. If anything, paste it from Word to a text editor like Notepad, so the crazy Microsoft formatting code is cleaned. It’s futile. People just use Word.

Here’s the weird thing.

Sometimes, people see a pop-up window in MailChimp that offers to “clean their Microsoft code from all the junk.

People have thanked us for it. Some people tell us it sucks, because it never truly cleans all the code. These people demand that we fix the “MailChimp Word Cleaner.”

But honestly, we had no friggin’ clue what they’re all talking about. We’ve never built such a tool.

Apparently, it’s something built into the WYSIWYG editor that we’re using. It’s a handy tool, but we’ve never seen it in action.

We’ve tried copy-pasting Word content all day, and it never popped up this “Word Cleaner” window. We tried different browsers, different operating systems, different versions of Word, different types and amounts of content. Nothing. We could never replicate what our users were reporting.

Aarron called it the “Mythical Unicorn Feature” that we could never catch.

One-handed operation

But today, while doing our on-site usability testing, our accountant was eating her asian chicken salad with one hand, and building an email campaign with the other:

That’s not our accountant. I just felt like using that stock photo, because it’s so ridiculous looking.

So instead of using keyboard shortcuts like CTRL+C (for copy) then CTRL+V (for paste), she was using the mouse’s right click button for everything.

We observed this:

Right-click –> Copy, then right-click –> Paste.

EUREKA:

We found the mythical unicorn!

Right-clicking is what triggered the Microsoft Word cleaner!

All this time, we were trying to replicate the problem by using keyboard shortcuts. No wonder we never found it.

By the way, it does an admirable job of cleaning Microsoft Word code, but it still leaves a few stray bits of junk code behind the scenes. Juuuust enough to screw up your HTML emails. So we STILL don’t recommend copy-pasting from Word.

Too much content?

If you find yourself copy-pasting from Microsoft Word a lot to make your email newsletters, is it because you have tons and tons of content to send?

Should the majority of that content be published on your blog or website instead? And perhaps your email should just be a quick blurb that invites subscribers to click over to your site to read the full article.

Some people just like to write big long articles, but some people just procrastinate. Or they don’t have time to log into MailChimp and build newsletters every single month. So their newsletter content builds up over time.

I can relate to that myself.

Which is why we created our RSS-to-email tool. You publish content and news to your blog. MailChimp takes that content, sticks it inside your HTML email newsletter template (all nice and pretty), then we send it to your subscribers. Automatically.

We discussed this with our accountant. She could setup a blog to post frequent updates. News, opinions on tax regulations, tips for organizing your business records and receipts, etc. And since it’s a blog, she can use it to show her personality. Then, once a quarter she can send the “big official newsletter” to her list, like she currently does in MailChimp. This approach, with the frequent blog updates in between official newsletters, has the added benefit of keeping her list clean and active.

Best of all, she may never have to open up Microsoft Word ever again.

_________

BTW, any MailChimp users in or kinda near Atlanta want a free lunch? Let us stop by your office and watch you work. We’ll hang around afterwards and talk email marketing with you, or show you how to do stuff in MailChimp. Great way to get some free consulting (and food).

Spread the monkey love:
  • TwitThis
  • Digg
  • Facebook
  • del.icio.us
  • Reddit
  • StumbleUpon
  • description
  • Google
  • LinkedIn
  • Ma.gnolia
  • MisterWong
  • Netvouz
  • NewsVine
  • Slashdot
  • Technorati
  • YahooMyWeb
  • BlinkList
  • Design Float
  • Mixx
  • Pownce
  • Propeller
  • Webnews.de

5 Comments

    • Jared says:

      Wow I have needed this… I always just pasted in something like notepad first.

    • Paul says:

      I had no idea that people creating email campaigns would not know about the Word copying no-no… wow.

      I would also urge those same people to be careful when copy-pasting from your website as a lot of times code will be transferred over as well. If you have text in a span the span tag will often copy over and could cause problems.

      I always paste either into my source code, or into notepad first like you have suggested.

    • philldo says:

      We have the same issue with people copying from Word to the WYSIWYG editor in our ecommerce CMS. We have told our staff about how to get around the bloat, but copy and pasting straight from Word is as you say it too natural an action to re-train.

      Love the RSS to email feature.

    • Josh says:

      TinyMCE, which I use for clients within Drupal, has an extra button which brings up a popup window that you can paste from Word into, and then IT strips the formatting crap (can I say that??). I was looking for a similar button within the MailChimp interface but couldn’t find it. Is there a similar function that you could implement?

    • Ben says:

      Thanks, Josh. A “clean word” button is certainly an option. I’d love nothing more than to clean ALL of that crap (yes, you can say crap here) out of a file.

      But then you’d have raw content with no formatting, which is a slightly inelegant solution for us, because our audience would balk at losing their bullet points and other “not quite crap” formatting.

      In this particular user test, the FCK WYSIWYG actually did a decent job of stripping bad stuff, at first glance. But upon closer inspection, we found it left a couple stray SPANS in the file that screwed with our user’s line-height in a couple paragraphs.

      Also, there was a completely stray, empty table just sitting in the middle of a paragraph. Since it was an empty table, it was invisible. So it looked like a bunch of dead space in her email content, and every time she hit delete to “move the paragraph up” it wouldn’t budge.

      At first, our user thought it was our WYSIWYG’s fault (MailChimp’s fault), but then she remembered that she once had a table in her original Word .doc right in that very spot, and she deleted it (or at least thought she did).

      See, the problem gets compounded by user error with the Word file itself.

      Perhaps I’m overthinking (I’ve been accused of doing that), but I couldn’t help but ask, “Why does one feel the need to write newsletter content with Microsoft Word in the first place?”

      I don’t think typing content into Word is a bad idea. I just feel the need to always ask “why?” in order to get to the bottom of the problem (and come up with new feature ideas).

      Why not type content directly into MailChimp? Yes, I do realize our interface doesn’t match MS Word exactly, but it’s extremely close. There are some buttons missing, but those are buttons that do things that you can’t do in HTML email. we specifically kept them out.

      Maybe our editor window is too small? Do people need more room to type?

      Should we provide a “toggle to full-screen” option (I’ve submitted that to our wishlist, BTW)?

      But wait—why do you need that much room to type? This is an email, not a novel we’re writing here.

      So I looked at the content our accountant was trying to send. It was basically a super-duper long checklist of “how to prep for 2008 taxes.”

      So we’re talking about someone who sends email once a year. Ah, so it’s basically a lot of content that’s built up over time. That’s why she needs to use Word. She’s compiling her content as she works on other stuff. Instead of logging in to MailChimp to put down her thoughts, she puts them into Word.

      And the fact that she uses MC once or twice a year makes it difficult for her, because we update our interface fairly often. So we get the “OMG I’m having some serious issues w/MC” calls.

      Lots of issues here. That’s what makes design so fun(strating).

      Ultimately, we helped her get the email done, and urged her to setup a blog, so that she could simply post her content there as the ideas popped into her brain. Then, she could use MailChimp’s RSS-to-email tool to send emails out every time she updates her blog.

      Plus, blogs are easy to update, and would allow her to show off a little of her personality (but not too much). They’re also easy to post links to relevant news, opinions on stuff, ideas, customer showcases, etc.

      Another added benefit to emailing her customers a little more frequently (via RSS to email) is her list will be cleaner.

      See, when you send once-or-twice-a-year emails, you have massive changes to your subscriber lists. People get laid off, they move, switch ISPs, etc. That leads to spam filtering and blocked IPs, which leads to more support calls for us.

      The root of the problem is really setting a nice pace for yourself. Don’t make the newsletter a huge project that takes you several months to send (I have this problem myself, BTW). Sending in smaller chunks avoids soooo many problems.

      See, I think too much.

      And this is exactly why we built the rss-to-email tool.

Leave a Reply

* indicated required
http://www.mailchimp.com/nonrestrictiveocean.php