Plus addressing on Sentora using Postfix

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Plus addressing on Sentora using Postfix

Philip McGaw
I am running Ubuntu Ubuntu 14.04.4 LTS “Trusty” and "postconf -d | grep mail_version” gives me “mail_version = 2.11.0”.

I have installed Sentora (http://sentora.org) which sets most of the configuration up for Postfix up, I have made some changes to allow me to use certificates for IMAP and SMTP SSL (https://skippy.org.uk/lets-encrypt-postfix-and-dovecot/),

My postfix config file is http://pastebin.com/vqBWhNM9, and Postfix master process configuration file is http://pastebin.com/1AUPiLSd.

I am trying to send an email to [hidden email] and [hidden email] (sending from an iCloud account), it works to [hidden email], the lines from mail.log are as follows:

> Mar 11 17:48:07 njoror postfix/smtpd[32706]: connect from mr11p26im-asmtp004.me.com[17.110.86.109]
> Mar 11 17:48:07 njoror postfix/smtpd[32706]: Anonymous TLS connection established from mr11p26im-asmtp004.me.com[17.110.86.109]: TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)
> Mar 11 17:48:08 njoror postfix/smtpd[32706]: 1BAF055AB6: client=mr11p26im-asmtp004.me.com[17.110.86.109]
> Mar 11 17:48:08 njoror postfix/cleanup[32710]: 1BAF055AB6: message-id=<[hidden email]>
> Mar 11 17:48:08 njoror postfix/qmgr[32599]: 1BAF055AB6: from=<[hidden email]>, size=1513, nrcpt=1 (queue active)
> Mar 11 17:48:08 njoror postfix/pipe[32605]: 1BAF055AB6: to=<[hidden email]>, relay=dovecot, delay=0.43, delays=0.34/0.01/0/0.09, dsn=4.1.1, status=SOFTBOUNCE (user unknown)
> Mar 11 17:48:08 njoror postfix/smtpd[32706]: disconnect from mr11p26im-asmtp004.me.com[17.110.86.109]

vs

> Mar 11 17:49:23 njoror postfix/smtpd[32706]: connect from mr11p26im-asmtp004.me.com[17.110.86.109]
> Mar 11 17:49:23 njoror postfix/smtpd[32706]: Anonymous TLS connection established from mr11p26im-asmtp004.me.com[17.110.86.109]: TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)
> Mar 11 17:49:24 njoror postfix/smtpd[32706]: 33D2355AB9: client=mr11p26im-asmtp004.me.com[17.110.86.109]
> Mar 11 17:49:24 njoror postfix/cleanup[32710]: 33D2355AB9: message-id=<[hidden email]>
> Mar 11 17:49:24 njoror postfix/qmgr[32599]: 33D2355AB9: from=<[hidden email]>, size=1249, nrcpt=1 (queue active)
> Mar 11 17:49:25 njoror postfix/smtpd[32706]: disconnect from mr11p26im-asmtp004.me.com[17.110.86.109]
> Mar 11 17:49:26 njoror postfix/pipe[32605]: 33D2355AB9: to=<[hidden email]>, relay=dovecot, delay=1.8, delays=0.63/0/0/1.2, dsn=2.0.0, status=sent (delivered via dovecot service)
> Mar 11 17:49:26 njoror postfix/qmgr[32599]: 33D2355AB9: removed


Looking at the lines from mail.log it looks like it thinks there should be a user called [hidden email], this has been puzzling me for a while, I was wondering if any one on here could see what was wrong, and what I needed to change, Looking at a few examples and online guides I thought I had covered all the bases.

signature.asc (243 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Plus addressing on Sentora using Postfix

Philip McGaw
Is there something I can add to this to help with getting an answer to what I need to do to get this working?

Sent from my iPhone

> On 6 Apr 2016, at 22:29, Philip McGaw <[hidden email]> wrote:
>
> I am running Ubuntu Ubuntu 14.04.4 LTS “Trusty” and "postconf -d | grep mail_version” gives me “mail_version = 2.11.0”.
>
> I have installed Sentora (http://sentora.org) which sets most of the configuration up for Postfix up, I have made some changes to allow me to use certificates for IMAP and SMTP SSL (https://skippy.org.uk/lets-encrypt-postfix-and-dovecot/),
>
> My postfix config file is http://pastebin.com/vqBWhNM9, and Postfix master process configuration file is http://pastebin.com/1AUPiLSd.
>
> I am trying to send an email to [hidden email] and [hidden email] (sending from an iCloud account), it works to [hidden email], the lines from mail.log are as follows:
>
>> Mar 11 17:48:07 njoror postfix/smtpd[32706]: connect from mr11p26im-asmtp004.me.com[17.110.86.109]
>> Mar 11 17:48:07 njoror postfix/smtpd[32706]: Anonymous TLS connection established from mr11p26im-asmtp004.me.com[17.110.86.109]: TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)
>> Mar 11 17:48:08 njoror postfix/smtpd[32706]: 1BAF055AB6: client=mr11p26im-asmtp004.me.com[17.110.86.109]
>> Mar 11 17:48:08 njoror postfix/cleanup[32710]: 1BAF055AB6: message-id=<[hidden email]>
>> Mar 11 17:48:08 njoror postfix/qmgr[32599]: 1BAF055AB6: from=<[hidden email]>, size=1513, nrcpt=1 (queue active)
>> Mar 11 17:48:08 njoror postfix/pipe[32605]: 1BAF055AB6: to=<[hidden email]>, relay=dovecot, delay=0.43, delays=0.34/0.01/0/0.09, dsn=4.1.1, status=SOFTBOUNCE (user unknown)
>> Mar 11 17:48:08 njoror postfix/smtpd[32706]: disconnect from mr11p26im-asmtp004.me.com[17.110.86.109]
>
> vs
>
>> Mar 11 17:49:23 njoror postfix/smtpd[32706]: connect from mr11p26im-asmtp004.me.com[17.110.86.109]
>> Mar 11 17:49:23 njoror postfix/smtpd[32706]: Anonymous TLS connection established from mr11p26im-asmtp004.me.com[17.110.86.109]: TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)
>> Mar 11 17:49:24 njoror postfix/smtpd[32706]: 33D2355AB9: client=mr11p26im-asmtp004.me.com[17.110.86.109]
>> Mar 11 17:49:24 njoror postfix/cleanup[32710]: 33D2355AB9: message-id=<[hidden email]>
>> Mar 11 17:49:24 njoror postfix/qmgr[32599]: 33D2355AB9: from=<[hidden email]>, size=1249, nrcpt=1 (queue active)
>> Mar 11 17:49:25 njoror postfix/smtpd[32706]: disconnect from mr11p26im-asmtp004.me.com[17.110.86.109]
>> Mar 11 17:49:26 njoror postfix/pipe[32605]: 33D2355AB9: to=<[hidden email]>, relay=dovecot, delay=1.8, delays=0.63/0/0/1.2, dsn=2.0.0, status=sent (delivered via dovecot service)
>> Mar 11 17:49:26 njoror postfix/qmgr[32599]: 33D2355AB9: removed
>
>
> Looking at the lines from mail.log it looks like it thinks there should be a user called [hidden email], this has been puzzling me for a while, I was wondering if any one on here could see what was wrong, and what I needed to change, Looking at a few examples and online guides I thought I had covered all the bases.

Reply | Threaded
Open this post in threaded view
|

Re: Plus addressing on Sentora using Postfix

Philip McGaw


> On 7 Apr 2016, at 21:51, Julio Cesar Covolato <[hidden email]> wrote:
>
>
> Em 07/04/2016 17:19, Philip McGaw escreveu:
>> Is there something I can add to this to help with getting an answer to what I need to do to get this working?
>>
>> Sent from my iPhone
>>
>>> On 6 Apr 2016, at 22:29, Philip McGaw <[hidden email]> wrote:
>>>
>>> I am running Ubuntu Ubuntu 14.04.4 LTS “Trusty” and "postconf -d | grep mail_version” gives me “mail_version = 2.11.0”.
>>>
>>> I have installed Sentora (http://sentora.org) which sets most of the configuration up for Postfix up, I have made some changes to allow me to use certificates for IMAP and SMTP SSL (https://skippy.org.uk/lets-encrypt-postfix-and-dovecot/),
>>>
>>> My postfix config file is http://pastebin.com/vqBWhNM9, and Postfix master process configuration file is http://pastebin.com/1AUPiLSd.
>>>
>>> I am trying to send an email to [hidden email] and [hidden email] (sending from an iCloud account), it works to [hidden email], the lines from mail.log are as follows:
>>>
>>>> Mar 11 17:48:07 njoror postfix/smtpd[32706]: connect from mr11p26im-asmtp004.me.com[17.110.86.109]
>>>> Mar 11 17:48:07 njoror postfix/smtpd[32706]: Anonymous TLS connection established from mr11p26im-asmtp004.me.com[17.110.86.109]: TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)
>>>> Mar 11 17:48:08 njoror postfix/smtpd[32706]: 1BAF055AB6: client=mr11p26im-asmtp004.me.com[17.110.86.109]
>>>> Mar 11 17:48:08 njoror postfix/cleanup[32710]: 1BAF055AB6: message-id=<[hidden email]>
>>>> Mar 11 17:48:08 njoror postfix/qmgr[32599]: 1BAF055AB6: from=<[hidden email]>, size=1513, nrcpt=1 (queue active)
>>>> Mar 11 17:48:08 njoror postfix/pipe[32605]: 1BAF055AB6: to=<[hidden email]>, relay=dovecot, delay=0.43, delays=0.34/0.01/0/0.09, dsn=4.1.1, status=SOFTBOUNCE (user unknown)
>>>> Mar 11 17:48:08 njoror postfix/smtpd[32706]: disconnect from mr11p26im-asmtp004.me.com[17.110.86.109]
>>> vs
>>>
>>>> Mar 11 17:49:23 njoror postfix/smtpd[32706]: connect from mr11p26im-asmtp004.me.com[17.110.86.109]
>>>> Mar 11 17:49:23 njoror postfix/smtpd[32706]: Anonymous TLS connection established from mr11p26im-asmtp004.me.com[17.110.86.109]: TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)
>>>> Mar 11 17:49:24 njoror postfix/smtpd[32706]: 33D2355AB9: client=mr11p26im-asmtp004.me.com[17.110.86.109]
>>>> Mar 11 17:49:24 njoror postfix/cleanup[32710]: 33D2355AB9: message-id=<[hidden email]>
>>>> Mar 11 17:49:24 njoror postfix/qmgr[32599]: 33D2355AB9: from=<[hidden email]>, size=1249, nrcpt=1 (queue active)
>>>> Mar 11 17:49:25 njoror postfix/smtpd[32706]: disconnect from mr11p26im-asmtp004.me.com[17.110.86.109]
>>>> Mar 11 17:49:26 njoror postfix/pipe[32605]: 33D2355AB9: to=<[hidden email]>, relay=dovecot, delay=1.8, delays=0.63/0/0/1.2, dsn=2.0.0, status=sent (delivered via dovecot service)
>>>> Mar 11 17:49:26 njoror postfix/qmgr[32599]: 33D2355AB9: removed
>>>
>>> Looking at the lines from mail.log it looks like it thinks there should be a user called [hidden email], this has been puzzling me for a while, I was wondering if any one on here could see what was wrong, and what I needed to change, Looking at a few examples and online guides I thought I had covered all the bases.
> And the correct way is send to: <[hidden email]>

[hidden email] is a real user. [hidden email] is the test of plus addressing failing.
Reply | Threaded
Open this post in threaded view
|

Re: Plus addressing on Sentora using Postfix

Philip McGaw
In reply to this post by Philip McGaw

> On 7 Apr 2016, at 21:44, Julio Cesar Covolato <[hidden email]> wrote:
>
>
> Em 07/04/2016 17:19, Philip McGaw escreveu:
>> Is there something I can add to this to help with getting an answer to what I need to do to get this working?
>>
>> Sent from my iPhone
>>
>>> On 6 Apr 2016, at 22:29, Philip McGaw <[hidden email]> wrote:
>>>
>>> I am running Ubuntu Ubuntu 14.04.4 LTS “Trusty” and "postconf -d | grep mail_version” gives me “mail_version = 2.11.0”.
>>>
>>> I have installed Sentora (http://sentora.org) which sets most of the configuration up for Postfix up, I have made some changes to allow me to use certificates for IMAP and SMTP SSL (https://skippy.org.uk/lets-encrypt-postfix-and-dovecot/),
>>>
>>> My postfix config file is http://pastebin.com/vqBWhNM9, and Postfix master process configuration file is http://pastebin.com/1AUPiLSd.
>>>
>>> I am trying to send an email to [hidden email] and [hidden email] (sending from an iCloud account), it works to [hidden email], the lines from mail.log are as follows:
>>>
>>>> Mar 11 17:48:07 njoror postfix/smtpd[32706]: connect from mr11p26im-asmtp004.me.com[17.110.86.109]
>>>> Mar 11 17:48:07 njoror postfix/smtpd[32706]: Anonymous TLS connection established from mr11p26im-asmtp004.me.com[17.110.86.109]: TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)
>>>> Mar 11 17:48:08 njoror postfix/smtpd[32706]: 1BAF055AB6: client=mr11p26im-asmtp004.me.com[17.110.86.109]
>>>> Mar 11 17:48:08 njoror postfix/cleanup[32710]: 1BAF055AB6: message-id=<[hidden email]>
>>>> Mar 11 17:48:08 njoror postfix/qmgr[32599]: 1BAF055AB6: from=<[hidden email]>, size=1513, nrcpt=1 (queue active)
>>>> Mar 11 17:48:08 njoror postfix/pipe[32605]: 1BAF055AB6: to=<[hidden email]>, relay=dovecot, delay=0.43, delays=0.34/0.01/0/0.09, dsn=4.1.1, status=SOFTBOUNCE (user unknown)
>>>> Mar 11 17:48:08 njoror postfix/smtpd[32706]: disconnect from mr11p26im-asmtp004.me.com[17.110.86.109]
>>> vs
>>>
>>>> Mar 11 17:49:23 njoror postfix/smtpd[32706]: connect from mr11p26im-asmtp004.me.com[17.110.86.109]
>>>> Mar 11 17:49:23 njoror postfix/smtpd[32706]: Anonymous TLS connection established from mr11p26im-asmtp004.me.com[17.110.86.109]: TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)
>>>> Mar 11 17:49:24 njoror postfix/smtpd[32706]: 33D2355AB9: client=mr11p26im-asmtp004.me.com[17.110.86.109]
>>>> Mar 11 17:49:24 njoror postfix/cleanup[32710]: 33D2355AB9: message-id=<[hidden email]>
>>>> Mar 11 17:49:24 njoror postfix/qmgr[32599]: 33D2355AB9: from=<[hidden email]>, size=1249, nrcpt=1 (queue active)
>>>> Mar 11 17:49:25 njoror postfix/smtpd[32706]: disconnect from mr11p26im-asmtp004.me.com[17.110.86.109]
>>>> Mar 11 17:49:26 njoror postfix/pipe[32605]: 33D2355AB9: to=<[hidden email]>, relay=dovecot, delay=1.8, delays=0.63/0/0/1.2, dsn=2.0.0, status=sent (delivered via dovecot service)
>>>> Mar 11 17:49:26 njoror postfix/qmgr[32599]: 33D2355AB9: removed
>>>
>>> Looking at the lines from mail.log it looks like it thinks there should be a user called [hidden email], this has been puzzling me for a while, I was wondering if any one on here could see what was wrong, and what I needed to change, Looking at a few examples and online guides I thought I had covered all the bases.
> Maibe you forgot setup "recipient_delimiter = +" on main.cf?

Set up, the main.cf file I have set up can be found here http://pastebin.com/vqBWhNM9
Reply | Threaded
Open this post in threaded view
|

Re: [+] Re: Plus addressing on Sentora using Postfix

Philip McGaw
In reply to this post by Philip McGaw
Thanks for the suggestion Julio,

I did the following: In my master.cf changing

> dovecot   unix  -       n       n       -       -       pipe    flags=DRhu user=vmail:mail argv=/usr/lib/dovecot/deliver -d ${recipient}


to

> dovecot unix    -       n       n       -       -      pipe     flags=DRhu user=vmail:mail argv=/usr/lib/dovecot/deliver -e -d ${recipient} -f ${sender}

(and then using service postfix restart) still results in

> Apr  7 22:31:55 njoror postfix/pipe[27183]: C190955AC8: to=<[hidden email]>, relay=dovecot, delay=0.23, delays=0.2/0/0/0.03, dsn=4.1.1, status=SOFTBOUNCE (user unknown)

In my log for emails to [hidden email], while [hidden email] gets delivered without problems.

is there anything else that I could be overlooking, or should be looking to share with [hidden email] to aid in getting a resolution, this has been causing me to wonder what is wrong for a while, as it should all work by now.

Regards

Philip “Skippy” McGaw

> On 7 Apr 2016, at 22:13, Julio Cesar Covolato <[hidden email]> wrote:
>
> In master.cf, try this dovecot setting:
>
> # Use dovecot deliver program as LDA.
> dovecot unix    -       n       n       -       -      pipe
>  flags=DRhu user=vmail:mail argv=/usr/lib/dovecot/deliver -e -d ${recipient} -f ${sender}
>
> -----------------------------
>    _    Engº Julio Cesar Covolato
>   0v0   <[hidden email]>
>  /(_)\  F: 55-11-3129-3366
>   ^ ^   PSI INTERNET
> -----------------------------
>
> Em 07/04/2016 17:53, Philip McGaw escreveu:
>>
>>> On 7 Apr 2016, at 21:51, Julio Cesar Covolato <[hidden email]> wrote:
>>>
>>>
>>> Em 07/04/2016 17:19, Philip McGaw escreveu:
>>>> Is there something I can add to this to help with getting an answer to what I need to do to get this working?
>>>>
>>>> Sent from my iPhone
>>>>
>>>>> On 6 Apr 2016, at 22:29, Philip McGaw <[hidden email]> wrote:
>>>>>
>>>>> I am running Ubuntu Ubuntu 14.04.4 LTS “Trusty” and "postconf -d | grep mail_version” gives me “mail_version = 2.11.0”.
>>>>>
>>>>> I have installed Sentora (http://sentora.org) which sets most of the configuration up for Postfix up, I have made some changes to allow me to use certificates for IMAP and SMTP SSL (https://skippy.org.uk/lets-encrypt-postfix-and-dovecot/),
>>>>>
>>>>> My postfix config file is http://pastebin.com/vqBWhNM9, and Postfix master process configuration file is http://pastebin.com/1AUPiLSd.
>>>>>
>>>>> I am trying to send an email to [hidden email] and [hidden email] (sending from an iCloud account), it works to [hidden email], the lines from mail.log are as follows:
>>>>>
>>>>>> Mar 11 17:48:07 njoror postfix/smtpd[32706]: connect from mr11p26im-asmtp004.me.com[17.110.86.109]
>>>>>> Mar 11 17:48:07 njoror postfix/smtpd[32706]: Anonymous TLS connection established from mr11p26im-asmtp004.me.com[17.110.86.109]: TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)
>>>>>> Mar 11 17:48:08 njoror postfix/smtpd[32706]: 1BAF055AB6: client=mr11p26im-asmtp004.me.com[17.110.86.109]
>>>>>> Mar 11 17:48:08 njoror postfix/cleanup[32710]: 1BAF055AB6: message-id=<[hidden email]>
>>>>>> Mar 11 17:48:08 njoror postfix/qmgr[32599]: 1BAF055AB6: from=<[hidden email]>, size=1513, nrcpt=1 (queue active)
>>>>>> Mar 11 17:48:08 njoror postfix/pipe[32605]: 1BAF055AB6: to=<[hidden email]>, relay=dovecot, delay=0.43, delays=0.34/0.01/0/0.09, dsn=4.1.1, status=SOFTBOUNCE (user unknown)
>>>>>> Mar 11 17:48:08 njoror postfix/smtpd[32706]: disconnect from mr11p26im-asmtp004.me.com[17.110.86.109]
>>>>> vs
>>>>>
>>>>>> Mar 11 17:49:23 njoror postfix/smtpd[32706]: connect from mr11p26im-asmtp004.me.com[17.110.86.109]
>>>>>> Mar 11 17:49:23 njoror postfix/smtpd[32706]: Anonymous TLS connection established from mr11p26im-asmtp004.me.com[17.110.86.109]: TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)
>>>>>> Mar 11 17:49:24 njoror postfix/smtpd[32706]: 33D2355AB9: client=mr11p26im-asmtp004.me.com[17.110.86.109]
>>>>>> Mar 11 17:49:24 njoror postfix/cleanup[32710]: 33D2355AB9: message-id=<[hidden email]>
>>>>>> Mar 11 17:49:24 njoror postfix/qmgr[32599]: 33D2355AB9: from=<[hidden email]>, size=1249, nrcpt=1 (queue active)
>>>>>> Mar 11 17:49:25 njoror postfix/smtpd[32706]: disconnect from mr11p26im-asmtp004.me.com[17.110.86.109]
>>>>>> Mar 11 17:49:26 njoror postfix/pipe[32605]: 33D2355AB9: to=<[hidden email]>, relay=dovecot, delay=1.8, delays=0.63/0/0/1.2, dsn=2.0.0, status=sent (delivered via dovecot service)
>>>>>> Mar 11 17:49:26 njoror postfix/qmgr[32599]: 33D2355AB9: removed
>>>>> Looking at the lines from mail.log it looks like it thinks there should be a user called [hidden email], this has been puzzling me for a while, I was wondering if any one on here could see what was wrong, and what I needed to change, Looking at a few examples and online guides I thought I had covered all the bases.
>>> And the correct way is send to: <[hidden email]>
>> [hidden email] is a real user. [hidden email] is the test of plus addressing failing.
>
>


signature.asc (243 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Plus addressing on Sentora using Postfix

Wietse Venema
In reply to this post by Philip McGaw
Philip McGaw:
> >> Mar 11 17:48:08 njoror postfix/pipe[32605]: 1BAF055AB6: to=<[hidden email]>, relay=dovecot, delay=0.43, delays=0.34/0.01/0/0.09, dsn=4.1.1, status=SOFTBOUNCE (user unknown)

Hint: >>>DOVECOT<<< says "user unknown:".

        Wietse

Reply | Threaded
Open this post in threaded view
|

Re: Plus addressing on Sentora using Postfix

Wietse Venema
Wietse Venema:
> Philip McGaw:
> > >> Mar 11 17:48:08 njoror postfix/pipe[32605]: 1BAF055AB6: to=<[hidden email]>, relay=dovecot, delay=0.43, delays=0.34/0.01/0/0.09, dsn=4.1.1, status=SOFTBOUNCE (user unknown)
>
> Hint: >>>DOVECOT<<< says "user unknown:".

As Viktor pointed out (I am not familiar with Dovecot configuration):

Dovecot documentation shows recommended command-line syntax for
the LDA:

        http://wiki.dovecot.org/LDA/Postfix

For example:

  dovecot unix    -       n       n       -       -      pipe
    flags=DRhu user=vmail:vmail
    argv=/usr/local/libexec/dovecot/dovecot-lda
    -f ${sender} -d ${user}@${nexthop} -m ${extension}

And correctly points out the need for:

   main.cf:
        dovecot_destination_recipient_limit = 1

Some users want to preserve case in the address extension:

   http://dovecot.org/pipermail/dovecot/2015-April/100503.html

Good luck.

        Wietse
Reply | Threaded
Open this post in threaded view
|

rate limiting

Casey Connor
Hello -- we're sending from a postfix instance as part of our stress-testing infrastructure.

We need to limit the rate of outbound messages per second: sometimes to a particular slow rate, sometimes to a medium speed, sometimes "as fast as possible".

I found no easy way to do that; have I missed something?

Our current method is to use a mail gun to send to postfix over localhost which tops up the queues periodically, and then use default_destination_concurrency_limit as a way to limit the number of concurrent outgoing clients (we're sending to only one domain for these tests).

This works well enough, but it's coarse, and I believe it's sensitive to changing network latencies.

We've considered an intermediate process between the mail gun and the local postfix instance to limit based on bandwidth, but I'm wondering if there is a smarter way we can manage this via postfix configs?

Thanks for any ideas!
-c

Konsole output
Reply | Threaded
Open this post in threaded view
|

Re: rate limiting

Wietse Venema
Reply | Threaded
Open this post in threaded view
|

Re: rate limiting

Casey Connor
Thank you -- will it accept decimal seconds?

We are sending on the order of 50-200+ messages per second in this
stress test, so the delay between messages could be smaller than .005
seconds.

On 04/07/2016 06:19 PM, Wietse Venema wrote:
> See:
> http://www.postfix.org/postconf.5.html#default_transport_rate_delay
> http://www.postfix.org/postconf.5.html#default_destination_rate_delay
>
> The names are similar but things work differently.
>
> Wietse

Reply | Threaded
Open this post in threaded view
|

Re: rate limiting

Viktor Dukhovni

> On Apr 7, 2016, at 11:32 PM, Casey Connor <[hidden email]> wrote:
>
> Thank you -- will it accept decimal seconds?

No.

> We are sending on the order of 50-200+ messages per second in this stress test, so the delay between messages could be smaller than .005 seconds.

Inject 50-200 messages per second into Postfix, and mail will not
be delivered faster than it comes in (unless a backlog develops
and clears when downstream conditions improve).

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: rate limiting

Casey Connor

>> We are sending on the order of 50-200+ messages per second in this stress test, so the delay between messages could be smaller than .005 seconds.
> Inject 50-200 messages per second into Postfix, and mail will not
> be delivered faster than it comes in (unless a backlog develops
> and clears when downstream conditions improve).

We don't have a tool to inject at a particular rate, but maybe we can
find one or make one. I was hoping we could handle it in postfix, but it
sounds like a no.

Please consider decimal-second rate limiting as a feature request. :-)

Thanks,
-c

Reply | Threaded
Open this post in threaded view
|

Re: rate limiting

Phil Stracchino
On 04/07/16 23:57, Casey Connor wrote:

>
>>> We are sending on the order of 50-200+ messages per second in this stress test, so the delay between messages could be smaller than .005 seconds.
>> Inject 50-200 messages per second into Postfix, and mail will not
>> be delivered faster than it comes in (unless a backlog develops
>> and clears when downstream conditions improve).
>
> We don't have a tool to inject at a particular rate, but maybe we can
> find one or make one. I was hoping we could handle it in postfix, but it
> sounds like a no.
>
> Please consider decimal-second rate limiting as a feature request. :-)

It's a pretty niche use case.  In a traffic-test environment such as
yours, it would seem the better approach is to simply have your test
tool control the rate at which it injects messages into Postfix.  The
message stream coming *out* of Postfix may occasionally not be 100%
uniform, but hey ...   *neither is the real world*.


--
  Phil Stracchino
  Babylon Communications
  [hidden email]
  [hidden email]
  Landline: 603.293.8485
Reply | Threaded
Open this post in threaded view
|

Re: rate limiting

Viktor Dukhovni
In reply to this post by Casey Connor

> On Apr 7, 2016, at 11:57 PM, Casey Connor <[hidden email]> wrote:
>
> Please consider decimal-second rate limiting as a feature request. :-)

It is very unlikely to happen.  To deliver much more than one message per
second, given typical message transaction latencies, delivery concurrency
is required, and simple message spacing strategies no longer work.

Achieving target rates is much more difficult than enforcing minimum
spacing.  Injecting at a target rate is easier, just create requests
at that rate and have some reasonable number of threads (10 to 20 or
so) submit messages in parallel via the Postfix server.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: rate limiting

Curtis Villamizar
In reply to this post by Casey Connor
In message <[hidden email]>
Casey Connor writes:
>
> Thank you -- will it accept decimal seconds?
>  
> We are sending on the order of 50-200+ messages per second in this
> stress test, so the delay between messages could be smaller than .005
> seconds.

If you are trying to simulate a very busy mailserver, then you should
be concerned about connections to it from multiple hosts per second
most sending one or a few messages and a realistic test would have to
have a nominal TCP RTT in the range of a few to a few 10s of msec
since the speed of light is limited and geographic delays come into
play.  I've been involved in testing and some simulation of this type
but on routers and various switchy-thingies rather than mailservers.

Curtis


> On 04/07/2016 06:19 PM, Wietse Venema wrote:
> > See:
> > http://www.postfix.org/postconf.5.html#default_transport_rate_delay
> > http://www.postfix.org/postconf.5.html#default_destination_rate_delay
> >
> > The names are similar but things work differently.
> >
> > Wietse
Reply | Threaded
Open this post in threaded view
|

Re: rate limiting

Casey Connor
Thanks, Curtis. We have taken all that in to consideration. I'll spare
you the long story, but we are testing somewhat specific things. :-) -c

> If you are trying to simulate a very busy mailserver, then you should
> be concerned about connections to it from multiple hosts per second
> most sending one or a few messages and a realistic test would have to
> have a nominal TCP RTT in the range of a few to a few 10s of msec
> since the speed of light is limited and geographic delays come into
> play. I've been involved in testing and some simulation of this type
> but on routers and various switchy-thingies rather than mailservers.
> Curtis