Using multiple instance, does it make sense?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Using multiple instance, does it make sense?

Christian Wittwer-2
Hi,
I'm trying to setup a postfix enviroment to relay about 200k mails per hour.
The hardware and the network link will be well dimensioned, so I like
to talk about the configuration part of this setup.
There are several options to tune in postfix, like timouts and
concurrent sessions etc..

My idea was basically to setup two smtp services.
One which has very short timeouts and uses lots of concurrent sessions etc..
A second one which takes the non deliverable mails from the first one
and tries with longer timeouts and less sessions to deliver mail.
So the first one is not blocked and can just continue to deliver, and
the second one makes the dirty work.
Does such a setup make sense to you?

I read about the ability to run multiple instances of postfix with
version > 2.6.
Would this be a setup for multiple instances?

Christian
Reply | Threaded
Open this post in threaded view
|

Re: Using multiple instance, does it make sense?

Barney Desmond
2009/7/30 Christian Wittwer <[hidden email]>:

> I'm trying to setup a postfix enviroment to relay about 200k mails per hour.
> The hardware and the network link will be well dimensioned, so I like
> to talk about the configuration part of this setup.
> There are several options to tune in postfix, like timouts and
> concurrent sessions etc..
>
> My idea was basically to setup two smtp services.
> One which has very short timeouts and uses lots of concurrent sessions etc..
> A second one which takes the non deliverable mails from the first one
> and tries with longer timeouts and less sessions to deliver mail.
> So the first one is not blocked and can just continue to deliver, and
> the second one makes the dirty work.
> Does such a setup make sense to you?

I have two immediate thoughts here; short one first:
How do you propose to move non-deliverable mail from the first
instance to the second one?

Actually, I have another small thought:
I know large mailouts have problems delivering some messages, that's a
fact of life. That said, can you "clean" your mailing list further? We
have some customers that do large mailouts to their subscriber list
and they point-blank refuse to remove bad addresses because they'd
have to admit to themselves that they don't have as many subscribers
as they claim to - I'm talking about hotmial.com and yahoo.coma.u, etc

The real point here:
To be honest, I reckon postfix is probably smarter than you. While
it's easy enough to come up with cases where you could micro-manage
postfix for optimal behaviour, it's had plenty of time to mature and
deal with plenty of corner-cases. Postfix is nominally I/O-bound, it
doesn't hang around when there's work to be done. Perhaps if you could
push the hard-to-deliver stuff off to another host, that'd relieve I/O
contention on the "easy deliveries", but this could well be premature
optimisation.

Postfix's scheduler attempts to be fair and make sure mail is
delivered evenly, so one destination doesn't swamp others. There was a
question about this recently, which referred to an older thread on the
topic. Relevant reading:
http://www.postfix.org/SCHEDULER_README.html
The recent thread:
http://www.phwinfo.com/forum/mailing-postfix-users/373193-re-message-priority.html
Which refers to this one from a few years ago:
http://www.irbs.net/internet/postfix/0407/2008.html

You mentioned your hardware is already spec'ed out? I'm mildly curious
what that looks like, as I published some very informal benchmark
figures a while ago, showing a modest system doing ~270,000 per hour
in contrived lab conditions with little optimisation.
Reply | Threaded
Open this post in threaded view
|

Re: Using multiple instance, does it make sense?

Christian Wittwer-2
2009/7/30 Barney Desmond <[hidden email]>:
> I have two immediate thoughts here; short one first:
> How do you propose to move non-deliverable mail from the first
> instance to the second one?

I think it should work with the parameter fallback_relay?
An other possiblity would be, to map directly volatile domains to the
second postfix with transport_map.

> Actually, I have another small thought:
> I know large mailouts have problems delivering some messages, that's a
> fact of life. That said, can you "clean" your mailing list further? We
> have some customers that do large mailouts to their subscriber list
> and they point-blank refuse to remove bad addresses because they'd
> have to admit to themselves that they don't have as many subscribers
> as they claim to - I'm talking about hotmial.com and yahoo.coma.u, etc

Yes, our mailing software automatically removes subscriber, which do
not exists anymore.
It can analyze the bounces messages, which goes to a pop mailbox.

> The real point here:
> To be honest, I reckon postfix is probably smarter than you. While
> it's easy enough to come up with cases where you could micro-manage
> postfix for optimal behaviour, it's had plenty of time to mature and
> deal with plenty of corner-cases. Postfix is nominally I/O-bound, it
> doesn't hang around when there's work to be done. Perhaps if you could
> push the hard-to-deliver stuff off to another host, that'd relieve I/O
> contention on the "easy deliveries", but this could well be premature
> optimisation.

Possible that postfix is smarter than me.. ;-)
The idea to use a second transport to deliver is from here:
http://www.linux.com/archive/feature/44901

> Postfix's scheduler attempts to be fair and make sure mail is
> delivered evenly, so one destination doesn't swamp others. There was a
> question about this recently, which referred to an older thread on the
> topic. Relevant reading:
> http://www.postfix.org/SCHEDULER_README.html
> The recent thread:
> http://www.phwinfo.com/forum/mailing-postfix-users/373193-re-message-priority.html
> Which refers to this one from a few years ago:
> http://www.irbs.net/internet/postfix/0407/2008.html

Thx for the links, interesting!

> You mentioned your hardware is already spec'ed out? I'm mildly curious
> what that looks like, as I published some very informal benchmark
> figures a while ago, showing a modest system doing ~270,000 per hour
> in contrived lab conditions with little optimisation.

I agree with you, that sending 200k mails per hour is not a real
problem with standard hardware.
At the moment I don't know the exact spec, but the customer said it
should be enough. If it's not, they have to upgrade the hardware ;-)
But I have to assure, that from the software side the setup is well
done and planed.

However I prefere the solution with just two smtp, a faster and a slower one.
I'm not sure whether if it should be two postfix instances or two smtp
transport.

What confuses me a bit at the moment. I read in the docs that postfix
starts with slow concurrency and then starts to increase these values.
But what happend, if the receiving mail-server blocks with "too many
sessions" error?
Does postfix then continue to send with a rate below this hard limit?
Maybe the second transport could then take these messages and send
with a much slower rate, to not swamp the other mail-server?
And if the second transport has a different ip (is this possible?)
then I would have not to wait until the "to man sessions" timeout is
over..

So, I'm really interested what you postfix guys do think about that?  ;-)

Christian