Syncing Database Changes from MailChimp to You:

In this scenario, you, the customer, has a database/list of sorts and you want to collect some or all of the information associated with your subscribers, preferably as it changes, and save it back to your database to do something with it.

Anywho, what might you want to do with that information you ask yourself? Well...

  1. If you rely on having valid email addresses for your customers, you'll definitely want to get bounce and unsubscribe data
  2. You may have important information in our Merge Fields or Interest Groups that your systems rely on somehow
  3. You want to know about new subscribers quickly and without parsing out emails
Before you even think about continuing, you are going to need:

Yup, I qualify... now what?

We'll start out by reminding you that these are just a few ways we've devised for accomplishing this. The crafty chimps in our labs and some of our savvy users often cook up new and ingenious ways of getting things done. By the way, if you are one of them, we'd love to hear your ideas... we might even send you a t-shirt or something!

Additionally, we STRONGLY discourage the practice of simply pulling your entire list, cycling though every user, and pulling down their details as a method of syncing up with us. It's an ugly, inefficient, brute force method of syncing this. That said, we understand that sometimes that is simply unavoidable...once, or maybe twice. And isn't it much more enjoyable to implement a unique, smart, and scalable solution? We think so, so we hope you keep reading.

By and far one of the most requested functions people ask about are ways to capture Subscribes, Unsubscribes, and Profile Updates in a real-time manner. As of April 1st, 2009 (no, it's not a joke) we support two standard, durable methods for syncing data and have documented two other methods that are sometimes useful for performing syncs via the API:

Webhooks

Webhooks essentially allow you to subscribe to various events occurring in our systems and have your systems receive notifications about them in near real-time. While we could go on and on about how they work here, we've already documented them separately, so follow that link and read through that.

API time/delta based syncs

As of the v1.2 release of our API, we now support some much smarter, more durable methods for obtaining only the updated data that you need via the API.

Keeping a List Synced

Keeping an internal database/list synced with your MailChimp list is by and far the type of sychronization that users are interested in and currently doing. As such after v1.2 and later releases of our API support an additional since parameter for the listMembers method. Using that parameter you can simply request all changes for a particular status since a certain date and time which typically would be the last time you ran your sync scripts. We also added an additional status, updated. Here's a recap of the statuses -

  • subscribed - retrieve all subscribed users since a particular date/time
  • unsubscribed - retrieve all users who have unsubscribed since a particular date/time
  • cleaned - retrieve all cleaned users since a particular date/time
  • updated - retrieve all users who have updated their profile data since a particular date/time
There are two caveats to be aware of when only using the API sync methods:
  1. Times for status changes for all items can come via a subscriber manually using your forms, an admin user logged in to our web interface, list imports, and even API calls/updates. If you want more granular control, you may want to consider using our Webhooks in addition to or instead of these methods.
  2. Changes to emails will not be easy to see (they will appear in the updated status list). To be sure you capture these, you may want to consider also using our Webhooks.

Keeping Stats and Reports Synced

Many organizations like to pull in stats and reports from their Campaigns and Lists to do cool things with them internally. There are now a few extra API methods and updates to help you keep with these tasks

  • campaigns() allows you to simply pass in a specific Campaign Id to get details for rather than having to basically do a hunt and pack search
  • Our Campaign Stats Methods are greatly expanded and give you access to pretty much all the stats and reports we have on your campaign.
  • Our Campaign AIM Methods let you get at the real hairy details of everything that happened.



Faking a Callback

Note:   You must switch your forms to advanced customization mode to use this!
Sometimes we call this the fake image hack, too. Anywho, here's what you do:

For each of those items that should be captured we will start by editing the "finished" form for each by:

  • logging into the MailChimp Web app
  • select the appropriate list
  • then click the design signup forms & response emails link there.
Data to TrackForm to Edit
SubscribesConfirmation "Thank You" Page
UnsubscribesUnsubscribe Success Page
Profile UpdatesUpdate Profile "Thank You" Page

What you are going to do is stick a fake image tag with some merge tags in the URL, probably near the bottom of the page and out of the way. Here are some examples:

<IMG SRC="URL=http://www.example.org/sub?email=*|EMAIL|*" height="1" width="1">

<IMG SRC="URL=http://www.example.org/unsub?email=*|EMAIL|*"> height="1" width="1">

<IMG SRC="http://www.example.org/profile?email=*|EMAIL|*"> height="1" width="1">

Depending on what you need, you can use any merge tags you have created or even our unique email ids (*|UNIQID|*) when creating that URL.

Ok, now on your server you will need to create the script(s) that will read in the variable(s) and update your system(s) appropriately. While you don't have to, we highly suggest returning valid image data just to be sure it display correctly in all browsers - some will ignore the tag if there is a not valid image, others (like IE) may place a nasty placeholder in your form - ew. Of course that image can (and probably should) be incredibly small and blank so the user doesn't even notice it.

In case you didn't get what we just did, putting that image tag in there makes a request back to your server. Sure it's a little hacky and weird to do that, but you know what? It works!. In fact, if you look around, there are lots of systems that keep site and use stats using that exact same method.

Finally, you may also want to consider adding some logic on your side to handle the inevitable cases where a user accidentally hits a page repeatedly and you don't want or should not keep making changes on your system.

Meta Redirects

Note:   You must switch your forms to advanced customization mode to use this!
These are simple and usually pretty obvious to folks. They also require the most effort on your side. Basically all you are going to do is edit your "start" form for each by:
  • logging into the MailChimp Web app
  • select the appropriate list
  • then click the design signup forms & response emails link there.
Here are the forms that what you'll be editing:
Data to TrackForm to Edit
SubscribesSignup Form
UnsubscribesUnsubscribe Page
Profile UpdatesUpdate Profile Form

The one complaint most people have with this method is that slow connections may see some of the page load before their browser actually does the redirect. The best we can tell you to do is to remove, comment, and hide as much as possible on the page and put the redirect at the very top. In case you are not familiar with meta redirects, here's some documentation. Basically you will stick a tag like one of these in there:

<META HTTP-EQUIV="refresh" content="0;URL=http://www.example.org/sub">

<META HTTP-EQUIV="refresh" content="0;URL=http://www.example.org/unsub?email=*|EMAIL|*">

<META HTTP-EQUIV="refresh" content="0;URL=http://www.example.org/profile?email=*|EMAIL|*">

Then you have them on your site and you have their email address. At this point you will essentially be responsible for replicating those forms and submitting the appropriate API call to us: