apocryph.org Notes to my future self

Finally moved my whole life's email into Google

In the beginning, I used Eudora Lite to check my POP3 mail accounts. Later came Pegasus Mail. I dabbled with Outlook Express. Some time around 1997 I settled definitively on Outlook, mostly due to it’s (then unique) support for multiple mail accounts each with its own identity, and a powerful filtering system that let me sort messages by the account they came in on.

It was around this time that I started another quirky practice: I never deleted email. Spam, lists, malware; whatever. I filed it in elaborate hierarchies, sure, but I never deleted any email. Ever.

I used Outlook in its various incarnations for years after that. Finally, in the Summer of 2005, I started to have my fill. My mailbox files were well in excess of two gigabytes, at a time when Outlook wasn’t expected to scale even to one. I had thousands upon thousands of emails, and searching them took forever. I also had my mailbox file stored on a network share of a machine that was slowly failing, so I kept getting mailbox corruption. Last, and worst of all, I was being buried in a flurry of spam.

And so it was that I set up a colo hosting account at CI Host, bought a 1U rackmount server, and filled it up with VMWare GSX Server images, including an OpenBSD 3.8 image which I set up to run my email. I had Postfix, IMAP, procmail, and SpamAssassin going. It was great, except for two problems:

1. How do get my Outlook mail into my new mail system

and

1. Spam filtering still wasn’t very good

At the time, Mozilla Thunderbird had the ability to suck mail out of an Outlook file and into itself, and from there via IMAP I could get it on the server, but I found that in the process any rich text and attachments didn’t make it, making the conversion function worthless.

Believe it or not, I ended up writing my own migration tool in C#, using a shareware MAPI .NET wrapper called MAPI33 and an open-source RFC822 writer library called OpenSmtp.net, combined with alot of my own code, to dump all the mail from my Outlook mailbox files into a Maildir filesystem, which I then rsyncd up to my email server.

Astonishingly, this worked rather well, and most importantly, preserved the rich text and attachment fidelity of my messages.

However, I never did find an adequate solution for spam filtering, despite using SpamAssassin extensively.

I put up with this until around June of 06, after I’d come back from Iraq using only a shitty SquirrelMail web interface to my email and convinced I had to find another solution. As it happened, my request to participate in the Google Apps for Domains beta was approved around that time, so I set up Gmail on my apocryph.org domain. And finally, I was home. Unless you’ve used shitty webmail interfaces like SquirrelMail, you can’t possibly imagine how wonderful Gmail’s interface is. And the spam filtering! NOTHING gets through.

So I happily went along with my email run by Google but on my domain. Then, a week ago, I discovered that sometime in the intervening year, Google had rolled out an option to import your old mail with IMAP. I could finally have ten years of email, all easily accessible and searchable via Gmail’s awesome web interface! So I rsyncd my Maildir up to my Dreamhost account, enabled IMAP on it, pointed Google’s IMAP importer at it, and two days later, 130,000 emails had been imported, catapulting my mail box size from < 500 MB to 2300 MB (good thing Google upped the quota on Apps for Domains accounts to 4GB!).

At last, I have a email system that is exactly what I want. I have an IMAP interface so I can run thick-client mail programs and have offline access to my email, I have a first-class awesome web interface, spam protection is rock-solid, and I can search and sort a huge body of mail without breaking a sweet. It was a long time coming, but it was worth the wait.

Now, please God don’t let Google become evil so I have to move my email somewhere else!