Limit rate/concurrency to a given domain

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Limit rate/concurrency to a given domain

Santiago Romero-2

 Hi.

 I have a postfix mailserver working as a secondary MX of a domain
(let's call it "domain.com"). The primary MX is a qmail+antivir+antispam
server that has serious memory/cpu problems to process all the email
(the one which receives by itself and the email that I sent to it).

 Is there any way to slow down the mx2 -> mx1 connection
concurrence/message ratio only for a given domain or for a single remote
mail server? I just want to limit as much as possible the mail
concurrency from my server to a given remote server to avoid affecting
its performance.

I've read in the documentation about "*_destination_concurrency_limit",
but they seem "global" parameters. I just want to act over a single
remote domain or server hostname/ip.

 Thanks for any help.

--
Santiago Romero
Reply | Threaded
Open this post in threaded view
|

Re: Limit rate/concurrency to a given domain

Sahil Tandon
On Feb 19, 2009, at 11:44 AM, Santiago Romero  
<[hidden email]> wrote:

>
> Hi.
>
> I have a postfix mailserver working as a secondary MX of a domain  
> (let's call it "domain.com"). The primary MX is a qmail+antivir
> +antispam server that has serious memory/cpu problems to process all  
> the email (the one which receives by itself and the email that I  
> sent to it).
>
> Is there any way to slow down the mx2 -> mx1 connection concurrence/
> message ratio only for a given domain or for a single remote mail  
> server? I just want to limit as much as possible the mail  
> concurrency from my server to a given remote server to avoid  
> affecting its performance.
>
> I've read in the documentation about  
> "*_destination_concurrency_limit", but they seem "global"  
> parameters. I just want to act over a single remote domain or server  
> hostname/ip.

The destination_concurrency_limit and rate_delay parameters can be  
configured per transport.  You could configure a transport map to  
route all mail destined to domain.com via the example: transport,  
which can be configured as a clone of the smtp transport in  
master.cf.  Then, specify concurrency and other limits/delays for only  
that transport in main.cf.

--
Sahil Tandon
Reply | Threaded
Open this post in threaded view
|

Re: Limit rate/concurrency to a given domain

Santiago Romero-2

>
> The destination_concurrency_limit and rate_delay parameters can be
> configured per transport.  You could configure a transport map to
> route all mail destined to domain.com via the example: transport,
> which can be configured as a clone of the smtp transport in
> master.cf.  Then, specify concurrency and other limits/delays for only
> that transport in main.cf.
>

 This way?

/etc/postfix/master.cf:
slow      inet  n       -       -       -       -       smtpd

/etc/postfix/transports:
domain1.com    slow:
domain2.com    slow:

/etc/postfix/main.cf:
slow_destination_concurrency_limit = 1
slow_destination_rate_delay = 1

 
 If I understood it correctly, it should be something like that, isn't it?

--
Santiago Romero


Reply | Threaded
Open this post in threaded view
|

Re: Limit rate/concurrency to a given domain

Santiago Romero-2

 Sorry, I meant "smtp" instead of "smtpd" in the master.cf.

>
> /etc/postfix/master.cf:
> slow      inet  n       -       -       -       -       smtp
>
> /etc/postfix/transports:
> domain1.com    slow:
> domain2.com    slow:
>
> /etc/postfix/main.cf:
> slow_destination_concurrency_limit = 1
> slow_destination_rate_delay = 1

 One question: is it also possible to do this instead of using main.cf?:

slow      inet  n       -       -       -       -       smtpd
 -o destination_concurrency_limit = 1,
 -o destination_rate_delay = 1


 Would that work without affecting the default
destination_concurrency_limit or rate_delay parameters for the standard
"smtp" transport?

 Thanks :)

--
Santiago Romero


Reply | Threaded
Open this post in threaded view
|

Re: Limit rate/concurrency to a given domain

Wietse Venema
Santiago Romero:
[ Charset ISO-8859-1 unsupported, converting... ]

>
>  Sorry, I meant "smtp" instead of "smtpd" in the master.cf.
> >
> > /etc/postfix/master.cf:
> > slow      inet  n       -       -       -       -       smtp
> >
> > /etc/postfix/transports:
> > domain1.com    slow:
> > domain2.com    slow:
> >
> > /etc/postfix/main.cf:
> > slow_destination_concurrency_limit = 1
> > slow_destination_rate_delay = 1
>
>  One question: is it also possible to do this instead of using main.cf?:
>
> slow      inet  n       -       -       -       -       smtpd
>  -o destination_concurrency_limit = 1,
>  -o destination_rate_delay = 1
>
>
>  Would that work without affecting the default
> destination_concurrency_limit or rate_delay parameters for the standard
> "smtp" transport?
>

default_destination_rate_delay (default: 0s)
       The default amount of delay that is inserted between individual  deliv-
       eries  to  the same destination; with per-destination recipient limit >
       1, a destination is a domain, otherwise it is a recipient.
..
       NOTE: THE DELAY IS ENFORCED BY THE QUEUE MANAGER. ...

The delay is NOT implemented by the smtpd process.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Limit rate/concurrency to a given domain

Santiago Romero-2
Wietse Venema escribió:
> default_destination_rate_delay (default: 0s)
>        The default amount of delay that is inserted between individual  deliv-
>        eries  to  the same destination; with per-destination recipient limit >
>        1, a destination is a domain, otherwise it is a recipient.
> ..
>        NOTE: THE DELAY IS ENFORCED BY THE QUEUE MANAGER. ...
>
> The delay is NOT implemented by the smtpd process.
>  
 I finally was setting:

/etc/postfix/master.cf:
slow      inet  n       -       -       -       -       smtp

/etc/postfix/transports:
domain1.com    slow:

/etc/postfix/main.cf:
slow_destination_recipient_limit = 1
slow_destination_concurrency_limit = 1
slow_destination_rate_delay = 2

 
 Do you mean (sorry If I didn't understood it) that I can use:

slow_destination_recipient_limit = 1
slow_destination_concurrency_limit = 1

 but no:

slow_destination_rate_delay = 2

 ?

 The logfile shows in any way that I'm sending a give email with the
"slow" transport instead of the standard "smtp" transport? (To verify if
my config is working correctly and check if emails are going out by the
right transport).

 Thanks for your help.

--
Santiago Romero


Reply | Threaded
Open this post in threaded view
|

Re: Limit rate/concurrency to a given domain

Wietse Venema
Santiago Romero:

> Wietse Venema escribi?:
> > default_destination_rate_delay (default: 0s)
> >        The default amount of delay that is inserted between individual  deliv-
> >        eries  to  the same destination; with per-destination recipient limit >
> >        1, a destination is a domain, otherwise it is a recipient.
> > ..
> >        NOTE: THE DELAY IS ENFORCED BY THE QUEUE MANAGER. ...
> >
> > The delay is NOT implemented by the smtpd process.
> >  

Therefore, "smtpd -o slow_destination_rate_delay" does not work,
nor does "smtp -o slow_destination_rate_delay".

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Limit rate/concurrency to a given domain

Santiago Romero-2
In reply to this post by Wietse Venema

 Is there any way to verify seeing the logfile if a given message is
going throught a given transport (say "slow" in my master.cf) instead of
standard "smtp" transport?

 I'm not sure or I don't know how to check if the messages are being
delivered by the "slow" transport I defined :? (at least, I can't see in
the logs info about it).

 Thanks.

--
Santiago Romero


Reply | Threaded
Open this post in threaded view
|

Re: Limit rate/concurrency to a given domain

Santiago Romero-2
In reply to this post by Wietse Venema

> Therefore, "smtpd -o slow_destination_rate_delay" does not work,
> nor does "smtp -o slow_destination_rate_delay".
>  

 So, how do I set an specific rate delay for a given "transport"?

 This way?:

qmgr      fifo  n       -       n       300     1       qmgr
   -o slow_destination_rate_delay =1

 ?

 Thanks.

--
Santiago Romero


Reply | Threaded
Open this post in threaded view
|

Re: Limit rate/concurrency to a given domain

Wietse Venema
Santiago Romero:

>
> > Therefore, "smtpd -o slow_destination_rate_delay" does not work,
> > nor does "smtp -o slow_destination_rate_delay".
> >  
>
>  So, how do I set an specific rate delay for a given "transport"?
>
>  This way?:
>
> qmgr      fifo  n       -       n       300     1       qmgr
>    -o slow_destination_rate_delay =1
>
>  ?

Put it in main.cf.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Limit rate/concurrency to a given domain

mouss-4
In reply to this post by Santiago Romero-2
Santiago Romero a écrit :
>
> Is there any way to verify seeing the logfile if a given message is
> going throught a given transport (say "slow" in my master.cf) instead of
> standard "smtp" transport?
>
> I'm not sure or I don't know how to check if the messages are being
> delivered by the "slow" transport I defined :? (at least, I can't see in
> the logs info about it).
>


look for "relay=".
Reply | Threaded
Open this post in threaded view
|

Re: Limit rate/concurrency to a given domain

Santiago Romero-2
mouss escribió:
>> Is there any way to verify seeing the logfile if a given message is
>> going throught a given transport (say "slow" in my master.cf) instead of
>> standard "smtp" transport?
>>
>> I'm not sure or I don't know how to check if the messages are being
>> delivered by the "slow" transport I defined :? (at least, I can't see in
>> the logs info about it).
> look for "relay="

 It just says:

Feb 22 10:15:28 egeo postfix/smtp[3679]: A0553B41AB: to=<[hidden email]>,
                                   relay=mail.domain.es[A.B.C.D]:25,
delay=13,
                                   delays=2.2/0/0.05/11, dsn=2.0.0,
                                   status=sent (250 ok 1235294128 qp 25569)

 It does not say anything about my "slow" transport, but I've noticed
that the destination_rate is working because email sent to
"mail.domain.es" is being "spaced" 2 seconds between a message and the
next one :-?

 What should the "relay=" field say when sending through a "new"
transport? :?

 It seems to be working but I don't see any difference in the log files
when I compare with the standar "smtp" transport :?

 Thanks a lot.

--
Santiago Romero


Reply | Threaded
Open this post in threaded view
|

Re: Limit rate/concurrency to a given domain

Noel Jones-2
Santiago Romero wrote:

> mouss escribió:
>>> Is there any way to verify seeing the logfile if a given message is
>>> going throught a given transport (say "slow" in my master.cf) instead of
>>> standard "smtp" transport?
>>>
>>> I'm not sure or I don't know how to check if the messages are being
>>> delivered by the "slow" transport I defined :? (at least, I can't see in
>>> the logs info about it).
>> look for "relay="
>
> It just says:
>
> Feb 22 10:15:28 egeo postfix/smtp[3679]: A0553B41AB:
> to=<[hidden email]>,
>                                   relay=mail.domain.es[A.B.C.D]:25,
> delay=13,
>                                   delays=2.2/0/0.05/11, dsn=2.0.0,
>                                   status=sent (250 ok 1235294128 qp 25569)
>
> It does not say anything about my "slow" transport, but I've noticed
> that the destination_rate is working because email sent to
> "mail.domain.es" is being "spaced" 2 seconds between a message and the
> next one :-?
>
> What should the "relay=" field say when sending through a "new"
> transport? :?
>
> It seems to be working but I don't see any difference in the log files
> when I compare with the standar "smtp" transport :?
>
> Thanks a lot.
>

That's right, the logs show the program name (smtp) not the
transport name; you can't tell which transport called smtp.

You can add something like
   -o syslog_name=postfix-slow
to the master.cf slow transport entry to differentiate the
logging.

   -- Noel Jones

Reply | Threaded
Open this post in threaded view
|

Re: Limit rate/concurrency to a given domain

Santiago Romero-2

> That's right, the logs show the program name (smtp) not the transport
> name; you can't tell which transport called smtp.
>
> You can add something like
>   -o syslog_name=postfix-slow
> to the master.cf slow transport entry to differentiate the logging.

 That's it. Postfix is awesome and very well designed. Such kind of
change would need 723435 patches in qmail (and that's why I'm migrating
all my qmail servers to postfix).

 Thanks a lot.

--
Santiago Romero


Reply | Threaded
Open this post in threaded view
|

Re: Limit rate/concurrency to a given domain

/dev/rob0
On Mon February 23 2009 09:52:26 Santiago Romero wrote:

> > That's right, the logs show the program name (smtp) not the
> > transport name; you can't tell which transport called smtp.
> >
> > You can add something like
> >   -o syslog_name=postfix-slow
> > to the master.cf slow transport entry to differentiate the logging.
>
>  That's it. Postfix is awesome and very well designed. Such kind of
> change would need 723435 patches in qmail (and that's why I'm
> migrating all my qmail servers to postfix).
>
>  Thanks a lot.

To be fair, probably only 723434; qmail logging is all done to stdout,
and you're free to set up any kind of filter or log preprocessor which
can do what you need. You still wouldn't have the clarity and plain
language information that Postfix logs offer, of course.
--
    Offlist mail to this address is discarded unless
    "/dev/rob0" or "not-spam" is in Subject: header