Migrating from qmail & BincIMAP to Exim and Dovecot on a Debian Sarge box
So, I've been meaning to do this for about nine months now. I have a Debian box at home which is, in fact, an old PowerMac 7300. It has fans, and makes noise. Since we moved to London the box has had to live on the bookshelf in the lounge. It takes up excessive amounts of space and is noisy, so the components for a fanless mini-itx system were bought, and constructed. But I haven't gotten around to migrating things over until now for many reasons, most of which concerned lack of spare time and the conceptual difference in handling virtual mail hosts delivering to local users.
I was running qmail + BincIMAP, which has worked for several years. My problem with this setup is that, given that I'm moving to another physical box, did I really want to have to go through the hassle of patching and compiling another qmail install. Why the shouldn't I use a server that's in Debian and get the free security updates. I'd initially chosen qmail because it was very very good. Now I want something that's good enough, but is supported and upgraded as part of my linux distro, which is something that qmail will never be.
Debian's default SMTP server is Exim, and I thought that, you know, the Debian folks are clever, so why don't I just use that. For IMAP server the choice isn't so clear. Binc was mostly fine, but I've had weirdness recently (unable to create folder children of INBOX). I've wrestled with Courier before, so no thanks, and no one seems to have much good to say about UW-IMAP. Some friends have been talking up Dovecot, and it's in Debian Sarge, so what the hell.
I've bought the Exim book, and it seems thorough and comprehensive, although it's unfortunately not a patch on Dave Sill's excellent The qmail handbook. Dovecot seems pretty straightforward to set up, so the big sticking patch was my virtual domains setup.
My qmail virtualdomains setup looks like this:
reprocessed.org:matthew-reprocessed
example@reprocessed.org:example-reprocessed
Where mails to everywhere except example@reprocessed.org are handled by the .qmail-reprocessed dot-qmail file hierarchy in my home dir, and example@reprocessed.org is handled by the .qmail-reprocessed dot-qmail file in example's home dir. My dot-qmail hierarchy .qmail-reprocessed-matt, .qmail-reprocessed-default, and .qmail-reprocessed-list-matt deal with anything to matt@, list-matt@ and anything_else@ reprocessed. Further processing (list subscriptions, really) is done by maildrop, which is invoked by the relevant dot-qmail file.
The trick is to replicate (or improve) on this using Exim. A key improvement for me would be to separate out some of the chunks of my mailbox hierarchy (mailing lists, one of my other domains) into separate IMAP accounts. Ideally, I'd still be able to get the kind of control I have with my current dot-qmail-and-maildrop setup.
At first this seemed pretty impossible, but after some poking and prodding I realised that I can do both.
The trick is to make use of Exim's router setup to dump mail in appropriate places, and then use a Dovecot passwd-file authentication setup to read a passwd-style file which matches IMAP requests with authentication credentials, but also the local uid & gid the mail IMAP process should run with. Not forgetting that Dovecot and Exim have to agree on where the appropriate places for mail are...
The setup allows me to have exim filters shunt mail around the place running as one local user, and several virtual IMAP users mapped to the local uid so that those mail stores can be accessed and modified by Dovecot. The Exim router setup means that I can have username-suffix format addresses mapped to the right local uid, so that I can have a hierarchy of .forward files (.forward, .forward-matt@example.org et al) in my home dir that can dispatch messages to several maildirs, each of which maps to a different Dovecot virtual user.
I've chosen to use a maildir domain/user hierarchy starting at /var/mail So, an example.
Using a Exim filter based dot-forward file, the following addresses are mapped to
local user matt and processed so that:
matt@example.org delivers to /var/mail/example.org/matt/Maildir
matt-suffix@example.org delivers to /var/mail/example.org/matt-suffix/Maildir
matt@another.example.com delivers to /var/mail/another.example.com/matt/Maildir
All these directories must be owned by local user matt, since the delivery process is running as matt.
Dovecot treats each directory as a separate IMAP account, and so I can use its authentication setup to create a user for each account which dovecot maps to the correct
uid and gid when the IMAP process runs.
It may sound a bit complicated, and it's probably not a scalable option, but since I run a mail server which has two users, that's not a big issue. In fact, it's not very complex, and is very flexible. And, using Exim's built-in filtering, I can eliminate the need to use Maildrop/Procmail totally.
There are two useful chunks of the Exim configuration that deal with this, first the userforward and local_user routers.
# Passes addresses through local user .forward Exim filters. Will look for
# ~user/.forward-user-suffix@host files (if the address has a suffix (i.e.
# isn't a local user)
userforward:
driver = redirect
check_local_user
local_part_suffix = -*
local_part_suffix_optional
file = $home/.forward-$local_part$local_part_suffix@$domain
allow_filter
no_verify
no_expn
check_ancestor
file_transport = address_file
directory_transport = address_dir
pipe_transport = address_pipe
reply_transport = address_reply
# Matches local user mailboxes. If the router fails, the error message is
# "Unknown address".
localuser:
driver = accept
check_local_user
local_part_suffix = -*
local_part_suffix_optional
transport = local_delivery
cannot_route_message = Unknown address
And then the a selection of transports, which handle the delivery. The default case (no dot-forward) deliveries are handled by the local_delivery transport, while address_file, address_dir and address_pipe are called when the Exim filters in the dot-forward perform one of their 'save' type commands (saving to a file, directory, or piping via an external command).
# handles delivery to maildir in /var/mail/DOMAIN/USER/Maildir
local_delivery:
driver = appendfile
directory = /var/mail/$domain/$local_part$local_part_suffix/Maildir
maildir_format
delivery_date_add
envelope_to_add
return_path_add
# This transport is used for handling pipe deliveries generated by alias or
# .forward files. If the pipe generates any standard output, it is returned
# to the sender of the message as a delivery error. Set return_fail_output
# instead of return_output if you want this to happen only when the pipe fails
# to complete normally. You can set different transports for aliases and
# forwards if you want to - see the references to address_pipe in the routers
# section above.
address_pipe:
driver = pipe
return_output
# This transport is used for handling deliveries directly to files that are
# generated by aliasing or forwarding.
address_file:
driver = appendfile
delivery_date_add
envelope_to_add
return_path_add
# This transport is used for handling deliveries directly to maildirs that are
# generated by aliasing or forwarding.
address_dir:
driver = appendfile
maildir_format
delivery_date_add
envelope_to_add
return_path_add
The relevant chunks of Dovecot config look like this:
default_mail_env = maildir:/var/mail/%d/%n/Maildir
Where, %n is the equivalent of Exim's $local_part and %d is the equivalent of $domain. And, for the user authentication:
auth_mechanisms = plain
auth_userdb = passwd-file /etc/dovecot/imap.passwd
auth_passdb = passwd-file /etc/dovecot/imap.passwd
The imap.passwd file looks something like this:
matt@example.org:CRYPTED_PASS_HERE:1000:1000::/var/mail/example.org:/bin/false::
matt-lists@example.org:CRYPTED_PASS_HERE:1000:1000::/var/mail/example.org:/bin/false::
Which tells Dovecot to run IMAP processes as uid and gid 1000 for both matt@example.org and matt-lists@example.org.
And, um, that's it. It works! The next step is to make use of the SA-Exim local_scan() patch to allow the use of SpamAssassin to reject spam at SMTP time with Exim.