multi pronged upgrade/migration

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

multi pronged upgrade/migration

Stan Hoeppner
This question touches on far more than Postfix, so I hope it doesn't offend
anyone being slightly OT.

A little background may help:

I built my current Postfix host in late 2005 as a single purpose
firewall/gateway to sit in front of an old Ms Exch server, on which I've
archived email since 2000.  The Postfix firewall has functioned in this mode for
4+ years.  It's running on a Debian system up to date at 5.0.3 with all patches
installed and a custom kernel, 2.6.31.1.  I'm running the Debian Lenny Postfix
2.5.5-1-1 package which I upgraded within the past few of months from 2.3.8
during the upgrade to Lenny.  I never configured local mail delivery on this
host as I installed Postfix on build day in '05 and set it to relay only.  I
have no experience with standard *nix mailbox or maildir formats, but I've
recently read up a little bit on the basics of both at the Dovecot website.

What I want to do is:

1.  Switch Postfix from relay mode to local delivery
2.  Convert entries in relay_recipients to aliases pointing to a local Unix user
3.  Select an appropriate mailbox format, location, and filessytem
4.  Install an appropriate IMAP/POP server daemon package
5.  Migrate current emails and folders to the new Postfix/IMAP combo
6.  What have I overlooked?

I know basically "how" to perform change [1] but the details depend on [2/3],
which I need advice on.  From what I've read on this list I think I'd like to
use Dovecot for IMAP.  What is the best way to reconfigure Postfix to drop mail
in the best format and location for Dovecot (mbox vs maildir)?  My current
Postfix spool is on / which has ~30GB free, ext2 FS.  If preferable I can create
a new partition formatted with any appropriate filesystem, mount that and have
Postfix/Dovecot dump new mail there, along with storing the IMAP mail and
folders.  I've already migrated most of the mail and folders from the old IMAP
server to local Thunderbird folders on my PC, so migrating those to the new IMAP
server should be fairly straight forward once I have the new IMAP server up and
running.

Of note, I have quite a few current IMAP folders with a thousand emails in each,
and one folder with over 10,000 emails.  Would mbox or maildir format be better
here, and does this effect Postfix in any way, or only the IMAP server?

Thanks for any/all advice, and again, sorry if some of this if OT.  I know many
here run Dovecot and other IMAP servers alongside Postfix so I figured you might
have some pointers.

Happy new year to all.

--
Stan
Reply | Threaded
Open this post in threaded view
|

Re: multi pronged upgrade/migration

Barney Desmond
I should warn beforehand that I'm probably not giving this as much
consideration as would be ideal - that'd mean breaking out a pen and
paper to collect my thoughts, which is a tough effort at this time of
year :)

2010/1/1 Stan Hoeppner <[hidden email]>:
> What I want to do is:
>
> 1.  Switch Postfix from relay mode to local delivery
> 2.  Convert entries in relay_recipients to aliases pointing to a local Unix user
> 3.  Select an appropriate mailbox format, location, and filessytem
> 4.  Install an appropriate IMAP/POP server daemon package
> 5.  Migrate current emails and folders to the new Postfix/IMAP combo
> 6.  What have I overlooked?

> I know basically "how" to perform change [1] but the details depend on [2/3],
> which I need advice on.

Your list of steps looks pretty solid, and they should be mostly
independent of each other. I don't have `postconf -n` (I'm sure you've
posted one to the list in recent history though), but I'll assume a
fairly simple and sane config with relay domains and recipients all
setup and ready to go.

The way we do things at work is to leave $mydestination at its default
value (only the machine's hostname and localhost are considered to be
local domains). For any "real" mail we're interested in, we then put
everything into virtual_alias_{domains,maps} - we're lazy and use bare
system usernames on the RHS of the maps for ease of administration.
This way, legitimate addresses get rewritten to something@$myorigin
(which defaults to $myhostname anyway, and is thus a local domain),
and handled as a local delivery. For your conversion, I would change
all your relay_domains to virtual_alias_domains, then add
virtual_alias_maps entries as you see fit.

N.B.:
1. I'm ignoring how you choose to go about creating the local system users
2. Things are a little messier if you only want to switch *some*
addresses in a domain to local delivery instead of relaying, but it's
doable.

> From what I've read on this list I think I'd like to use Dovecot for IMAP.
> What is the best way to reconfigure Postfix to drop mail
> in the best format and location for Dovecot (mbox vs maildir)?
> My current
> Postfix spool is on / which has ~30GB free, ext2 FS.  If preferable I can create
> a new partition formatted with any appropriate filesystem, mount that and have
> Postfix/Dovecot dump new mail there, along with storing the IMAP mail and
> folders.

I reckon you're almost in Holy War territory here. Like all good
answers, "it depends", and there's no shortage of people telling you
why their setup is best. A quick list of thoughts on filesystems and
formats (not all necessarily accurate)
* ext2 is stable, mature, supported everywhere, pretty good performance
* ext3 adds journals, yay for integrity! just as stable, mature and
supported as ext2. If you have last millenium-era hardware it might
perform worse. *sob*
* reiserfs, tail-packing, great for small files like maildir!
** (but it'll murder your data...)
* XFS has great performance, especially for deleting lots of files.
BTW I hope you have battery-backed write caches
* ZFS is new and shiny and will bring about world peace; pity it'll
never be in mainline, but hey, you can always use it with FUSE...
* butterfs isn't stable yet, it might land face-down with all your data
* mbox is historically crap for scaled performance due to its monolithic nature
* Maildir might solve world hunger, but thousands of small files could
get inefficient
* store mail in /var/mail/$username - this should be the default for
mboxes anyway
* store mail in ~/Maildir - a popular choice for maildirs
* If these users are only being used for mail, maybe you want a
separate mail partition after all; it could make
administration/backups/policy/whatever easier for you. It's easy
enough to specify your own delivery location

I think you can see where I'm going with this. Go with what you're
most comfortable with, then come back if you're actually having
performance problems. I'm a sysadmin for a webhosting company, and we
care mostly about stability (but not so much that we think it's okay
that Redhat ships postfix 2.3). One way to answer this is to ask
yourself, do YOU want to be the one who has to handle a 12hr repair
job when something blows up on a Sunday because you went for a more
"interesting" setup choice. Not saying it doesn't happen with stable
choices either, it's about what you're confident with.

As for mbox vs. maildir, it depends on usage. mbox is just fine for
infrequently-used or infrequently-updated mailboxes. Note that postfix
has a builtin mailbox_size_limit of 50MiB. If your mboxes are getting
that big, it's a hint that you probably want to be using maildir.
Maildir is great for busy mailboxes, but all those files have their
own cost too.

> Of note, I have quite a few current IMAP folders with a thousand emails in each,
> and one folder with over 10,000 emails.  Would mbox or maildir format be better
> here, and does this effect Postfix in any way, or only the IMAP server?

That doesn't sound unreasonable for maildir, but it's too variable to
give a firm answer either way. If you're unsure, I suspect most people
will recommend maildir, whatever the shortcomings. I believe the scope
of effect should be mostly on the IMAP server, but if the IMAP server
has an mbox locked then postfix might block on the delivery (I'm not
certain on this point though).

One thing I'd consider is using Dovecot as your local delivery agent
(it's imaginatively called "deliver"). It's flexible with respect to
mailbox formats, and can update its own indexes while it makes the
delivery. Dovecot is also a great IMAP server, you'll find plenty of
supporters for it here.

You mentioned migrating in emails and folders; this will be the most
frustrating and time-consuming part of the whole plan, the earlier
steps are probably only several hours of work. Seeing as you said this
box has only been used for relaying before, I'm assuming you want to
migrate everything from the Exchange server. Mail migrations are never
fun, so I wish you good luck with that. Assuming you sort out the
customer-facing side of things, you should be able to do it with a
small downtime window. I'm thinking you could enable soft_bounce then
flick your configuration from relay to local (once all the local
accounts are created). imapsync can be used to shuffle mail to the
postfix machine, then you fix up the networking once it's all done
(DNS change, IP migration, or tell users to update their mail
clients). Unless I've missed something, the worst case should be that
users can't see new mail.