Quantcast

No milters have been used at around midnight

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

No milters have been used at around midnight

Christian Rößner
Hi,

this morning I found a spam mail in my inbox, which normally should have been triggered by my spam milter. As I checked the headers, I found out that the milter service did not add any headers.

I checked the logs for the QID and found out that the milter was not even requested. Further I saw that not even one milter was requested:

Mar 30 00:02:20 mx postfix/postscreen[20916]: PASS NEW [2a02:4a8:ac24:126::105:130]:53402
Mar 30 00:02:20 mx postfix/smtpd[20918]: connect from ipv6.antirelay.smtp.cz[2a02:4a8:ac24:126::105:130]:53402
Mar 30 00:02:22 mx postfix/smtpd[20918]: Anonymous TLS connection established from ipv6.antirelay.smtp.cz[2a02:4a8:ac24:126::
105:130]:53402: TLSv1 with cipher ADH-AES256-SHA (256/256 bits)
Mar 30 00:02:22 mx postfix/smtpd[20918]: 3vthZQ6rwlzGp4v: client=ipv6.antirelay.smtp.cz[2a02:4a8:ac24:126::105:130]:53402
Mar 30 00:02:22 mx postfix/incoming/cleanup[20926]: 3vthZQ6rwlzGp4v: message-id=<[hidden email]>
Mar 30 00:02:23 mx postfix/qmgr[4629]: 3vthZQ6rwlzGp4v: from=<[hidden email]>, size=57769, nrcpt=1 (queue active)
Mar 30 00:02:23 mx postfix/smtpd[20918]: disconnect from ipv6.antirelay.smtp.cz[2a02:4a8:ac24:126::105:130]:53402 ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7
Mar 30 00:02:23 mx postfix/lmtp[20920]: 3vthZQ6rwlzGp4v: to=<[hidden email]>, orig_to=<[hidden email]>, relay=::1[::1]:24, delay=0.32, delays=0.19/0.01/0.01/0.11, dsn=2.0.0, status=sent (250 2.0.0 <[hidden email]> GTYCBe8u3FjsUQAAm3ipfw Saved)
Mar 30 00:02:23 mx postfix/qmgr[4629]: 3vthZQ6rwlzGp4v: removed

There exists only one exception for turning off milters which is shown here:

smtpd_milter_maps:
---------------------------------------------------------------
# relay.roessner-net.de
134.255.226.249                         DISABLE
[2a05:bec0:28:1:134:255:226:249]        DISABLE
---------------------------------------------------------------

Unfortunately I do not know how to reproduce this issue. I do not understand why none of the milters where requested.

There does not exist any special treatment for milters (say exceptions, whatever) for milters except the server "relay" as shown above.

Here is a part of the main.cf that handles the milters:

main.cf:
---------------------------------------------------------------
vrfydmn_opposite = {
    inet:[::1]:30074,
    connect_timeout=5s,
    default_action=accept
    }
spammilter = {
    inet:[::1]:30076,
    connect_timeout=5s,
    default_action=accept
    }

milter_connect_macros =
    j,
    v,
    {client_ptr},
    {daemon_name},
    {daemon_addr},
    {daemon_port}

milter_mail_macros =
    i,
    {auth_type},
    {auth_authen},
    {auth_author},
    {mail_addr},
    {mail_host},
    {mail_mailer},
    {client_name}

incoming_smtpd_milters =
    ${vrfydmn_opposite},
    ${spammilter}
---------------------------------------------------------------

master.cf:
---------------------------------------------------------------
smtpd     pass  -       -       y       -       -       smtpd
    -o smtpd_milters=${incoming_smtpd_milters}
    -o cleanup_service_name=cleanup2
---------------------------------------------------------------

As you see in the logs, there are no connect messages from both milters.

The setup is unchanged since months. The only thing that I could guess is:

- this spam is around midnight. At the same time (1-2mins difference), other connections from "relay.roessner-net.de" and "mail.roessner-net.de" came in and worked as expected. Daily logroate stuff. "relay" would switch off milters...
- Last "foreign" mail (not one of my own servers) sent a mail with working milters at 23:45:22 and after 00:08:39

So the problem occurred when my relay server was active _and_ a remote MTA connected.

But: If the timestamps are correct in syslog, I did not have simultaneous mails at midnight. Just one-by-one. But several.

Any suggestions, if I miss something? Could this be a problem with smtpd_milters_maps that some switching did not work as expected? I have no idea :)

Btw: Postfix 3.2.0

Kind regards

Christian
--
Erlenwiese 14, 36304 Alsfeld
T: +49 6631 78823400, F: +49 6631 78823409, M: +49 171 9905345


smime.p7s (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No milters have been used at around midnight

Wietse Venema
Nothing was changed, but something stopped working. Do you have ECC
memory enabled?

        Wietse
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No milters have been used at around midnight

Christian Rößner

> Am 30.03.2017 um 16:00 schrieb Wietse Venema <[hidden email]>:
>
> Nothing was changed, but something stopped working. Do you have ECC
> memory enabled?

It is a VM, but the host uses ECC-RAM. No errors were reported to the kernel message buffer.

Christian
--
Erlenwiese 14, 36304 Alsfeld
T: +49 6631 78823400, F: +49 6631 78823409, M: +49 171 9905345


smime.p7s (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No milters have been used at around midnight

Viktor Dukhovni

> On Mar 30, 2017, at 11:15 AM, Christian Rößner <[hidden email]> wrote:
>
> It is a VM, but the host uses ECC-RAM. No errors were reported to the kernel message buffer.

Is it possible that some log messages were lost when the log
socket got re-created as part of log-rotation?

Do your milters always add headers?  Or only for spammy messages?

Also, you have default_action=accept, perhaps your milters were too
busy at the time.  When did milter logging cease?  When did it resume?
What was the message delivery rate during the "gap"...

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No milters have been used at around midnight

Christian Rößner
Hi,

> Am 30.03.2017 um 17:25 schrieb Viktor Dukhovni <[hidden email]>:
>
>
>> On Mar 30, 2017, at 11:15 AM, Christian Rößner <[hidden email]> wrote:
>>
>> It is a VM, but the host uses ECC-RAM. No errors were reported to the kernel message buffer.
>
> Is it possible that some log messages were lost when the log
> socket got re-created as part of log-rotation?

Unfortunately not. There was no load on the system. It is standard rsyslog.

> Do your milters always add headers?  Or only for spammy messages?

Yes and they also always log to syslog on connect/discconect and processing mails.

> Also, you have default_action=accept, perhaps your milters were too
> busy at the time.  When did milter logging cease?  When did it resume?
> What was the message delivery rate during the "gap"...

There have been very few mails at that time. That is what makes me wonder. No load could have caused this issue.

I personally wonder how many people make use of that new feature smtpd_milters_maps and maybe nobody else discovered this issue though.

Maybe I need to figure out how to do a test on sending mail from my relay server, while simultaneously sending mail from remote and watching. I think that should trigger the problem again, if I am right. Let's see what I can do here.


Christian
--
Erlenwiese 14, 36304 Alsfeld
T: +49 6631 78823400, F: +49 6631 78823409, M: +49 171 9905345


smime.p7s (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No milters have been used at around midnight

Christian Rößner
Hi,

> Am 31.03.2017 um 10:48 schrieb Christian Rößner <[hidden email]>:
>
> Hi,
>
>> Am 30.03.2017 um 17:25 schrieb Viktor Dukhovni <[hidden email]>:
>>
>>
>>> On Mar 30, 2017, at 11:15 AM, Christian Rößner <[hidden email]> wrote:
>>>
>>> It is a VM, but the host uses ECC-RAM. No errors were reported to the kernel message buffer.
>>
>> Is it possible that some log messages were lost when the log
>> socket got re-created as part of log-rotation?
>
> Unfortunately not. There was no load on the system. It is standard rsyslog.
>
>> Do your milters always add headers?  Or only for spammy messages?
>
> Yes and they also always log to syslog on connect/discconect and processing mails.
>
>> Also, you have default_action=accept, perhaps your milters were too
>> busy at the time.  When did milter logging cease?  When did it resume?
>> What was the message delivery rate during the "gap"...
>
> There have been very few mails at that time. That is what makes me wonder. No load could have caused this issue.
>
> I personally wonder how many people make use of that new feature smtpd_milters_maps and maybe nobody else discovered this issue though.
>
> Maybe I need to figure out how to do a test on sending mail from my relay server, while simultaneously sending mail from remote and watching. I think that should trigger the problem again, if I am right. Let's see what I can do here.
I have done tests and I can reproduce this issue _always_.

I did...:

for ((i=0; i<100; i++)); do ./foo.sh ; done

... on my mail server that runs the relay.roessner-net.de and the mx.roessner-net.de instances.

foo.sh looks like this:

-----------------------------------------------------------------------
#!/bin/bash

swaks \
-4 \
--ehlo "mx.roessner-net.de" \
-li 134.255.226.249 \
--server "relay.roessner-net.de" \
--from "[hidden email]" \
--to "[hidden email]" \
--h-Date "$(date -R)" \
--tls

exit 0
-----------------------------------------------------------------------

After I ran this (in fact I was too slow ;-) ), I sent a mail from external MTA mail.sys4.de and this mail did not run through any of the milters. It is much more worse than I thought, because each mail after that "mail loop" above was not scanned by any milters anymore! I also stopped postfix and started it again and still no milters got connected when sending test mails from external MTA. After that I sent a mail from my local Mac to my mail server, which also has a submission instance. That instance activated all milters it needed. And only after I did sent over the submission instance, the mx.roessner-net.de instance started using milters again.

That is a serious problem.

Something seems to cache the "disabled" state somewhere.

I have three instances:

postmulti -l
-----------------------------------------------------------------------
-               -               y         /etc/postfix
postfix-relay   -               y         /etc/postfix-relay
postfix-submission -               y         /etc/postfix-submission
-----------------------------------------------------------------------

The first instance receives mail from the internet and relays it to Dovecot. It also accepts mail from the submission instance and relays these mails to the internet.

The relay instance relays mails directly to the internet. It is a hub for all virtual servers and my web server. If mails are to myself, it also connects to mx.roessner-net.de but with milters disabled as shown in my first mail.

The submission instance is for authenticated users only and forwards all mail to the first instance.

So this is my setup that works for years now, until I requested the new smtpd_milters_maps feature. The issue started with Postfix 3.2

Hopefully I gave enough information to find the source of this issue.

Christian
--
Erlenwiese 14, 36304 Alsfeld
T: +49 6631 78823400, F: +49 6631 78823409, M: +49 171 9905345


smime.p7s (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No milters have been used at around midnight

Wietse Venema
> After I ran this (in fact I was too slow ;-) ), I sent a mail from
> external MTA mail.sys4.de and this mail did not run through any
> of the milters. It is much more worse than I thought, because each
> mail after that "mail loop" above was not scanned by any milters
> anymore! I also stopped postfix and started it again and still no
> milters got connected when sending test mails from external MTA.

If this problem persists after stopping and starting Postfix, then
you have proved that either it is not a Postfix problem or that your
measurement procedure is in error.

There is no POSTFIX Milter-related state that persists across
restarts, but there may be such state elsewhere: rate-limiting
policy servers, state in the kernel's networking stack, clients
deciding to deliver mail elsewhere, ...

        Wietse
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No milters have been used at around midnight

Wietse Venema
Wietse Venema:

> > After I ran this (in fact I was too slow ;-) ), I sent a mail from
> > external MTA mail.sys4.de and this mail did not run through any
> > of the milters. It is much more worse than I thought, because each
> > mail after that "mail loop" above was not scanned by any milters
> > anymore! I also stopped postfix and started it again and still no
> > milters got connected when sending test mails from external MTA.
>
> If this problem persists after stopping and starting Postfix, then
> you have proved that either it is not a Postfix problem or that your
> measurement procedure is in error.
>
> There is no POSTFIX Milter-related state that persists across
> restarts, but there may be such state elsewhere: rate-limiting
> policy servers, state in the kernel's networking stack, clients
> deciding to deliver mail elsewhere, ...

Especially relevant for logging is system-effing-d log rate limiting,
which reportedly has lower thresholds than rsyslog log rate limiting,
meaning that systemd throws away events before rsyslogd can log them.
This mechanism persists across Postfix restarts.

        Wietse
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No milters have been used at around midnight

Christian Rößner

> Am 03.04.2017 um 15:43 schrieb Wietse Venema <[hidden email]>:
>
> Wietse Venema:
>>> After I ran this (in fact I was too slow ;-) ), I sent a mail from
>>> external MTA mail.sys4.de and this mail did not run through any
>>> of the milters. It is much more worse than I thought, because each
>>> mail after that "mail loop" above was not scanned by any milters
>>> anymore! I also stopped postfix and started it again and still no
>>> milters got connected when sending test mails from external MTA.
>>
>> If this problem persists after stopping and starting Postfix, then
>> you have proved that either it is not a Postfix problem or that your
>> measurement procedure is in error.
>>
>> There is no POSTFIX Milter-related state that persists across
>> restarts, but there may be such state elsewhere: rate-limiting
>> policy servers, state in the kernel's networking stack, clients
>> deciding to deliver mail elsewhere, ...
>
> Especially relevant for logging is system-effing-d log rate limiting,
> which reportedly has lower thresholds than rsyslog log rate limiting,
> meaning that systemd throws away events before rsyslogd can log them.
> This mechanism persists across Postfix restarts.
I am not using systemd. (Gentoo Open-RC)

Also both milters add headers, which are missing in the received test mails. Which indicates that the milters did not run.

Also this is a very low traffic server. After doing the 100 mail loop, I sent three or four test mail with 30 second delays. No other mails passed the system. So no logger would suppress this.

I alo found out that sending from submission does not solve the problem. I repeated my test. It seems there exists some kind of timeout somewhere that is postfix stop/start aware. After that time period milters start working again. I only can guess: 2-5 minutes later milter begin working again.

I only have the Dovecot-Quota policy service (non else) and I have no other services that would cache things. Only two milters (vrfydmn and a anti-spam milter as shown in my first mail).

Christian
--
Erlenwiese 14, 36304 Alsfeld
T: +49 6631 78823400, F: +49 6631 78823409, M: +49 171 9905345


smime.p7s (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No milters have been used at around midnight

Christian Rößner

> Am 03.04.2017 um 16:07 schrieb Christian Rößner <[hidden email]>:
>
>>
>> Am 03.04.2017 um 15:43 schrieb Wietse Venema <[hidden email]>:
>>
>> Wietse Venema:
>>>> After I ran this (in fact I was too slow ;-) ), I sent a mail from
>>>> external MTA mail.sys4.de and this mail did not run through any
>>>> of the milters. It is much more worse than I thought, because each
>>>> mail after that "mail loop" above was not scanned by any milters
>>>> anymore! I also stopped postfix and started it again and still no
>>>> milters got connected when sending test mails from external MTA.
>>>
>>> If this problem persists after stopping and starting Postfix, then
>>> you have proved that either it is not a Postfix problem or that your
>>> measurement procedure is in error.
>>>
>>> There is no POSTFIX Milter-related state that persists across
>>> restarts, but there may be such state elsewhere: rate-limiting
>>> policy servers, state in the kernel's networking stack, clients
>>> deciding to deliver mail elsewhere, ...
>>
>> Especially relevant for logging is system-effing-d log rate limiting,
>> which reportedly has lower thresholds than rsyslog log rate limiting,
>> meaning that systemd throws away events before rsyslogd can log them.
>> This mechanism persists across Postfix restarts.
>
> I am not using systemd. (Gentoo Open-RC)
>
> Also both milters add headers, which are missing in the received test mails. Which indicates that the milters did not run.
>
> Also this is a very low traffic server. After doing the 100 mail loop, I sent three or four test mail with 30 second delays. No other mails passed the system. So no logger would suppress this.
>
> I alo found out that sending from submission does not solve the problem. I repeated my test. It seems there exists some kind of timeout somewhere that is postfix stop/start aware. After that time period milters start working again. I only can guess: 2-5 minutes later milter begin working again.
>
> I only have the Dovecot-Quota policy service (non else) and I have no other services that would cache things. Only two milters (vrfydmn and a anti-spam milter as shown in my first mail).
I did the opposite test now and disabled the smtpd_milter_maps parameter. After that I did the same tests again and milters work always. So enabling this map starts producing the described issue.

Christian
--
Erlenwiese 14, 36304 Alsfeld
T: +49 6631 78823400, F: +49 6631 78823409, M: +49 171 9905345


smime.p7s (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No milters have been used at around midnight

Christian Rößner

> Am 03.04.2017 um 16:47 schrieb Christian Rößner <[hidden email]>:
>
>>
>> Am 03.04.2017 um 16:07 schrieb Christian Rößner <[hidden email]>:
>>
>>>
>>> Am 03.04.2017 um 15:43 schrieb Wietse Venema <[hidden email]>:
>>>
>>> Wietse Venema:
>>>>> After I ran this (in fact I was too slow ;-) ), I sent a mail from
>>>>> external MTA mail.sys4.de and this mail did not run through any
>>>>> of the milters. It is much more worse than I thought, because each
>>>>> mail after that "mail loop" above was not scanned by any milters
>>>>> anymore! I also stopped postfix and started it again and still no
>>>>> milters got connected when sending test mails from external MTA.
>>>>
>>>> If this problem persists after stopping and starting Postfix, then
>>>> you have proved that either it is not a Postfix problem or that your
>>>> measurement procedure is in error.
>>>>
>>>> There is no POSTFIX Milter-related state that persists across
>>>> restarts, but there may be such state elsewhere: rate-limiting
>>>> policy servers, state in the kernel's networking stack, clients
>>>> deciding to deliver mail elsewhere, ...
>>>
>>> Especially relevant for logging is system-effing-d log rate limiting,
>>> which reportedly has lower thresholds than rsyslog log rate limiting,
>>> meaning that systemd throws away events before rsyslogd can log them.
>>> This mechanism persists across Postfix restarts.
>>
>> I am not using systemd. (Gentoo Open-RC)
>>
>> Also both milters add headers, which are missing in the received test mails. Which indicates that the milters did not run.
>>
>> Also this is a very low traffic server. After doing the 100 mail loop, I sent three or four test mail with 30 second delays. No other mails passed the system. So no logger would suppress this.
>>
>> I alo found out that sending from submission does not solve the problem. I repeated my test. It seems there exists some kind of timeout somewhere that is postfix stop/start aware. After that time period milters start working again. I only can guess: 2-5 minutes later milter begin working again.
>>
>> I only have the Dovecot-Quota policy service (non else) and I have no other services that would cache things. Only two milters (vrfydmn and a anti-spam milter as shown in my first mail).
>
> I did the opposite test now and disabled the smtpd_milter_maps parameter. After that I did the same tests again and milters work always. So enabling this map starts producing the described issue.
And some more tests and information:

I enabled the smptd_milter_maps again and triggered the problem again. After that I counted seconds. And while counting, I sent mails from remote each 5 seconds. I aborted after more than 6 minutes. Then I waited about 80 seconds and sent again and milters worked again. It seems the milters stay unused as long as there exists traffic.

So the system only starts using the milters again, if there is some silence on the system.

I think I have tested as good as I could. Hopefully you have enough information now.

Christian
--
Erlenwiese 14, 36304 Alsfeld
T: +49 6631 78823400, F: +49 6631 78823409, M: +49 171 9905345


smime.p7s (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No milters have been used at around midnight

Wietse Venema
In reply to this post by Christian Rößner
Christian Ro??ner:
> I did the opposite test now and disabled the smtpd_milter_maps
> parameter. After that I did the same tests again and milters work
> always. So enabling this map starts producing the described issue.

And the problem persists across stopping and starting Postfix,
therefore the problem is outside of Postfix. Good luck.

        Wietse
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No milters have been used at around midnight

Christian Rößner

> Am 03.04.2017 um 17:53 schrieb Wietse Venema <[hidden email]>:
>
> Christian Ro??ner:
>> I did the opposite test now and disabled the smtpd_milter_maps
>> parameter. After that I did the same tests again and milters work
>> always. So enabling this map starts producing the described issue.
>
> And the problem persists across stopping and starting Postfix,
> therefore the problem is outside of Postfix. Good luck.

Ok, if you are right, can you give me a last hint please. Do you think, this might be kernel related or something with iptables/conntrack?

I use a GRSecurity kernel. Also Shorewall, which enables conntrack. Could this be a source of this problem? If so, I will do several tests by using a different non-patched kernel and maybe disabling all conntrack stuff.

Any idea is welcome. Thanks

Christian
--
Erlenwiese 14, 36304 Alsfeld
T: +49 6631 78823400, F: +49 6631 78823409, M: +49 171 9905345


smime.p7s (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Patch: Postfix 3.3-20170218 (and 3.2) sticky disabled milters

Viktor Dukhovni
In reply to this post by Christian Rößner
On Mon, Apr 03, 2017 at 05:08:58PM +0200, Christian Rößner wrote:

> I enabled the smptd_milter_maps again and triggered the problem again.
> After that I counted seconds. And while counting, I sent mails from remote
> each 5 seconds. I aborted after more than 6 minutes. Then I waited about
> 80 seconds and sent again and milters worked again. It seems the milters
> stay unused as long as there exists traffic.
>
> So the system only starts using the milters again, if there is some silence on the system.
>
> I think I have tested as good as I could. Hopefully you have enough information now.

Yes, that's enough information.  Patch below:

diff --git a/src/smtpd/smtpd.c b/src/smtpd/smtpd.c
index 986264b2..1eaf1300 100644
--- a/src/smtpd/smtpd.c
+++ b/src/smtpd/smtpd.c
@@ -1461,6 +1461,7 @@ static NAMADR_LIST *hogger_list;
   * Other application-specific globals.
   */
 int     smtpd_input_transp_mask;
+static int smtpd_input_transp_mask_saved;
 
  /*
   * Forward declarations.
@@ -5386,6 +5387,8 @@ static void setup_milters(SMTPD_STATE *state)
      */
     if (state->milters == 0)
  smtpd_input_transp_mask |= INPUT_TRANSP_MILTER;
+    else
+ smtpd_input_transp_mask = smtpd_input_transp_mask_saved;
 }
 
 /* teardown_milters - release resources */
@@ -5714,7 +5717,7 @@ static void post_jail_init(char *unused_name, char **unused_argv)
      * Initialize the receive transparency options: do we want unknown
      * recipient checks, address mapping, header_body_checks?.
      */
-    smtpd_input_transp_mask =
+    smtpd_input_transp_mask_saved = smtpd_input_transp_mask =
     input_transp_mask(VAR_INPUT_TRANSP, var_input_transp);
 
     /*

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No milters have been used at around midnight

Viktor Dukhovni
In reply to this post by Christian Rößner

> On Apr 3, 2017, at 1:24 PM, Christian Rößner <[hidden email]> wrote:
>
> Any idea is welcome.

For Postfix 3.2.0 see: https://github.com/vdukhovni/postfix/pull/8
For 3.3-20170218, see: https://github.com/vdukhovni/postfix/pull/7

The underlying patch is identical.  There could be other ways
to address the issue, perhaps by avoiding changing the static
smtpd_input_transp_mask variable, but such a change would be
more invasive.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Patch: Postfix 3.3-20170218 (and 3.2) sticky disabled milters

Wietse Venema
In reply to this post by Viktor Dukhovni
Just for the record, Victor's finding falsifies the claim that the
problem, once triggered, persists across postfix stop/start.

I suggest moving the fix into the teardown_milters() function. It's
still not perfect, but it keeps things in one place.

        Wietse

*** ./src/smtpd/smtpd.c- 2017-02-18 20:58:21.000000000 -0500
--- ./src/smtpd/smtpd.c 2017-04-03 14:03:29.000000000 -0400
***************
*** 5396,5401 ****
--- 5396,5403 ----
  milter_free(state->milters);
  state->milters = 0;
      }
+     smtpd_input_transp_mask =
+ input_transp_mask(VAR_INPUT_TRANSP, var_input_transp);
  }
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No milters have been used at around midnight

Viktor Dukhovni
In reply to this post by Viktor Dukhovni
On Mon, Apr 03, 2017 at 02:03:41PM -0400, Viktor Dukhovni wrote:

> > Any idea is welcome.
>
> For Postfix 3.2.0 see: https://github.com/vdukhovni/postfix/pull/8
> For 3.3-20170218, see: https://github.com/vdukhovni/postfix/pull/7
>
> The underlying patch is identical.  There could be other ways
> to address the issue, perhaps by avoiding changing the static
> smtpd_input_transp_mask variable, but such a change would be
> more invasive.

On the other hand, perhaps not so bad after all:

diff --git a/src/smtpd/smtpd.c b/src/smtpd/smtpd.c
index 986264b2..742863c4 100644
--- a/src/smtpd/smtpd.c
+++ b/src/smtpd/smtpd.c
@@ -2007,8 +2007,15 @@ static int mail_open_stream(SMTPD_STATE *state)
     else if (SMTPD_STAND_ALONE(state) == 0) {
  int     cleanup_flags;
 
+        /*
+         * Safety: disable non_smtpd_milters when not sending our own mail
+         * filter list. Otherwise the next stage could handle this message as a
+         * local submission.
+         */
  cleanup_flags = input_transp_cleanup(CLEANUP_FLAG_MASK_EXTERNAL,
-     smtpd_input_transp_mask)
+     smtpd_input_transp_mask |
+                                             (state->milters ? 0 :
+                                              INPUT_TRANSP_MILTER))
     | CLEANUP_FLAG_SMTP_REPLY;
  if (state->flags & SMTPD_FLAG_SMTPUTF8)
     cleanup_flags |= CLEANUP_FLAG_SMTPUTF8;
@@ -5378,14 +5385,6 @@ static void setup_milters(SMTPD_STATE *state)
        var_milt_unk_macros,
        var_milt_macro_deflts);
     }
-
-    /*
-     * Safety: disable non_smtpd_milters when not sending our own mail filter
-     * list. Otherwise the next stage could handle this message as a local
-     * submission.
-     */
-    if (state->milters == 0)
- smtpd_input_transp_mask |= INPUT_TRANSP_MILTER;
 }
 
 /* teardown_milters - release resources */

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No milters have been used at around midnight

Christian Rößner
In reply to this post by Christian Rößner
Hi Victor,

do you have a patch that works for Postfix-3.2.0? I tried to apply the patch below, but it does not succeed.

Thanks

Christian

-------------------------

On Mon, Apr 03, 2017 at 02:03:41PM -0400, Viktor Dukhovni wrote:

> > Any idea is welcome.
>
> For Postfix 3.2.0 see: https://github.com/vdukhovni/postfix/pull/8
> For 3.3-20170218, see: https://github.com/vdukhovni/postfix/pull/7
>
> The underlying patch is identical.  There could be other ways
> to address the issue, perhaps by avoiding changing the static
> smtpd_input_transp_mask variable, but such a change would be
> more invasive.

On the other hand, perhaps not so bad after all:

diff --git a/src/smtpd/smtpd.c b/src/smtpd/smtpd.c
index 986264b2..742863c4 100644
--- a/src/smtpd/smtpd.c
+++ b/src/smtpd/smtpd.c
@@ -2007,8 +2007,15 @@ static int mail_open_stream(SMTPD_STATE *state)
     else if (SMTPD_STAND_ALONE(state) == 0) {
  int     cleanup_flags;
 
+        /*
+         * Safety: disable non_smtpd_milters when not sending our own mail
+         * filter list. Otherwise the next stage could handle this message as a
+         * local submission.
+         */
  cleanup_flags = input_transp_cleanup(CLEANUP_FLAG_MASK_EXTERNAL,
-    smtpd_input_transp_mask)
+    smtpd_input_transp_mask |
+                                             (state->milters ? 0 :
+                                              INPUT_TRANSP_MILTER))
    | CLEANUP_FLAG_SMTP_REPLY;
  if (state->flags & SMTPD_FLAG_SMTPUTF8)
    cleanup_flags |= CLEANUP_FLAG_SMTPUTF8;
@@ -5378,14 +5385,6 @@ static void setup_milters(SMTPD_STATE *state)
       var_milt_unk_macros,
       var_milt_macro_deflts);
     }
-
-    /*
-     * Safety: disable non_smtpd_milters when not sending our own mail filter
-     * list. Otherwise the next stage could handle this message as a local
-     * submission.
-     */
-    if (state->milters == 0)
- smtpd_input_transp_mask |= INPUT_TRANSP_MILTER;
 }
 
 /* teardown_milters - release resources */

--
        Viktor.

smime.p7s (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No milters have been used at around midnight

Wietse Venema
I posted two-line patch that should work with 3.1 and 3.2 (developed for 3.3).

        Wietse

Christian Ro??ner:

> Hi Victor,
>
> do you have a patch that works for Postfix-3.2.0? I tried to apply the patch below, but it does not succeed.
>
> Thanks
>
> Christian
>
> -------------------------
>
> On Mon, Apr 03, 2017 at 02:03:41PM -0400, Viktor Dukhovni wrote:
>
> > > Any idea is welcome.
> >
> > For Postfix 3.2.0 see: https://github.com/vdukhovni/postfix/pull/8
> > For 3.3-20170218, see: https://github.com/vdukhovni/postfix/pull/7
> >
> > The underlying patch is identical.  There could be other ways
> > to address the issue, perhaps by avoiding changing the static
> > smtpd_input_transp_mask variable, but such a change would be
> > more invasive.
>
> On the other hand, perhaps not so bad after all:
>
> diff --git a/src/smtpd/smtpd.c b/src/smtpd/smtpd.c
> index 986264b2..742863c4 100644
> --- a/src/smtpd/smtpd.c
> +++ b/src/smtpd/smtpd.c
> @@ -2007,8 +2007,15 @@ static int mail_open_stream(SMTPD_STATE *state)
>      else if (SMTPD_STAND_ALONE(state) == 0) {
>   int     cleanup_flags;
>  
> +        /*
> +         * Safety: disable non_smtpd_milters when not sending our own mail
> +         * filter list. Otherwise the next stage could handle this message as a
> +         * local submission.
> +         */
>   cleanup_flags = input_transp_cleanup(CLEANUP_FLAG_MASK_EXTERNAL,
> -    smtpd_input_transp_mask)
> +    smtpd_input_transp_mask |
> +                                             (state->milters ? 0 :
> +                                              INPUT_TRANSP_MILTER))
>     | CLEANUP_FLAG_SMTP_REPLY;
>   if (state->flags & SMTPD_FLAG_SMTPUTF8)
>     cleanup_flags |= CLEANUP_FLAG_SMTPUTF8;
> @@ -5378,14 +5385,6 @@ static void setup_milters(SMTPD_STATE *state)
>        var_milt_unk_macros,
>        var_milt_macro_deflts);
>      }
> -
> -    /*
> -     * Safety: disable non_smtpd_milters when not sending our own mail filter
> -     * list. Otherwise the next stage could handle this message as a local
> -     * submission.
> -     */
> -    if (state->milters == 0)
> - smtpd_input_transp_mask |= INPUT_TRANSP_MILTER;
>  }
>  
>  /* teardown_milters - release resources */
>
> --
>         Viktor.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: No milters have been used at around midnight

Christian Rößner

> Am 04.04.2017 um 12:58 schrieb Wietse Venema <[hidden email]>:
>
> I posted two-line patch that should work with 3.1 and 3.2 (developed for 3.3).

Applied, tested, works. Problem solved

Thanks

Christian

>
> Wietse
>
> Christian Ro??ner:
>> Hi Victor,
>>
>> do you have a patch that works for Postfix-3.2.0? I tried to apply the patch below, but it does not succeed.
>>
>> Thanks
>>
>> Christian
>>
>> -------------------------
>>
>> On Mon, Apr 03, 2017 at 02:03:41PM -0400, Viktor Dukhovni wrote:
>>
>>>> Any idea is welcome.
>>>
>>> For Postfix 3.2.0 see: https://github.com/vdukhovni/postfix/pull/8
>>> For 3.3-20170218, see: https://github.com/vdukhovni/postfix/pull/7
>>>
>>> The underlying patch is identical.  There could be other ways
>>> to address the issue, perhaps by avoiding changing the static
>>> smtpd_input_transp_mask variable, but such a change would be
>>> more invasive.
>>
>> On the other hand, perhaps not so bad after all:
>>
>> diff --git a/src/smtpd/smtpd.c b/src/smtpd/smtpd.c
>> index 986264b2..742863c4 100644
>> --- a/src/smtpd/smtpd.c
>> +++ b/src/smtpd/smtpd.c
>> @@ -2007,8 +2007,15 @@ static int mail_open_stream(SMTPD_STATE *state)
>>     else if (SMTPD_STAND_ALONE(state) == 0) {
>> int     cleanup_flags;
>>
>> +        /*
>> +         * Safety: disable non_smtpd_milters when not sending our own mail
>> +         * filter list. Otherwise the next stage could handle this message as a
>> +         * local submission.
>> +         */
>> cleanup_flags = input_transp_cleanup(CLEANUP_FLAG_MASK_EXTERNAL,
>> -    smtpd_input_transp_mask)
>> +    smtpd_input_transp_mask |
>> +                                             (state->milters ? 0 :
>> +                                              INPUT_TRANSP_MILTER))
>>   | CLEANUP_FLAG_SMTP_REPLY;
>> if (state->flags & SMTPD_FLAG_SMTPUTF8)
>>   cleanup_flags |= CLEANUP_FLAG_SMTPUTF8;
>> @@ -5378,14 +5385,6 @@ static void setup_milters(SMTPD_STATE *state)
>>      var_milt_unk_macros,
>>      var_milt_macro_deflts);
>>     }
>> -
>> -    /*
>> -     * Safety: disable non_smtpd_milters when not sending our own mail filter
>> -     * list. Otherwise the next stage could handle this message as a local
>> -     * submission.
>> -     */
>> -    if (state->milters == 0)
>> - smtpd_input_transp_mask |= INPUT_TRANSP_MILTER;
>> }
>>
>> /* teardown_milters - release resources */
>>
>> --
>>        Viktor.
Liebe Grüße

Christian
--
Erlenwiese 14, 36304 Alsfeld
T: +49 6631 78823400, F: +49 6631 78823409, M: +49 171 9905345


smime.p7s (2K) Download Attachment
Loading...