Postscreen vs. BDAT

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Postscreen vs. BDAT

Ralf Hildebrandt-2
Today a fellow postmaster (using Exim) called me, they were having problems sending
mail to charite.de. In my log I found:

Sep  3 00:31:18 mail-cbf postfix/postscreen[34943]: CONNECT from [31.7.179.105]:38256 to [193.175.73.208]:25
Sep  3 00:31:24 mail-cbf postfix/tlsproxy[39995]: CONNECT from [31.7.179.105]:38256
Sep  3 00:31:24 mail-cbf postfix/tlsproxy[39995]: Anonymous TLS connection established from [31.7.179.105]:38256: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Sep  3 00:31:24 mail-cbf postfix/postscreen[34943]: NOQUEUE: reject: RCPT from [31.7.179.105]:38256: 450 4.3.2 Service currently unavailable; from=<[hidden email]>, to=<[hidden email]>, proto=ESMTP, helo=<smtpgw03.nextwerk.de>
Sep  3 00:31:24 mail-cbf postfix/postscreen[34943]: COMMAND PIPELINING from [31.7.179.105]:38256 after BDAT: Received: from [213.182.136.75] (helo=NWH-EX13.nwcloud.loc)\r\n\tby smtpgw03.nextwerk.de with esmtpsa (

After I disabled chunking, mail would flow again.
Is this an Exim issue or a flaw in the newly introduced BDAT feature?

--
[*] sys4 AG

https://sys4.de, +49 (89) 30 90 46 64
Schleißheimer Straße 26/MG, 80333 München
                                           
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
Reply | Threaded
Open this post in threaded view
|

Re: Postscreen vs. BDAT

Viktor Dukhovni


> On Sep 3, 2018, at 8:41 AM, Ralf Hildebrandt <[hidden email]> wrote:
>
> Today a fellow postmaster (using Exim) called me, they were having problems sending
> mail to charite.de. In my log I found:
>
> Sep  3 00:31:18 mail-cbf postfix/postscreen[34943]: CONNECT from [31.7.179.105]:38256 to [193.175.73.208]:25
> Sep  3 00:31:24 mail-cbf postfix/tlsproxy[39995]: CONNECT from [31.7.179.105]:38256
> Sep  3 00:31:24 mail-cbf postfix/tlsproxy[39995]: Anonymous TLS connection established from [31.7.179.105]:38256: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
> Sep  3 00:31:24 mail-cbf postfix/postscreen[34943]: NOQUEUE: reject: RCPT from [31.7.179.105]:38256: 450 4.3.2 Service currently unavailable; from=<[hidden email]>, to=<[hidden email]>, proto=ESMTP, helo=<smtpgw03.nextwerk.de>
> Sep  3 00:31:24 mail-cbf postfix/postscreen[34943]: COMMAND PIPELINING from [31.7.179.105]:38256 after BDAT: Received: from [213.182.136.75] (helo=NWH-EX13.nwcloud.loc)\r\n\tby smtpgw03.nextwerk.de with esmtpsa (
>
> After I disabled chunking, mail would flow again.
> Is this an Exim issue or a flaw in the newly introduced BDAT feature?

The "postscreen" service intentionally does not support ESMTP PIPELINING:

posttls-finger "[mail-cbf-ext.charite.de]"
posttls-finger: Connected to mail-cbf-ext.charite.de[193.175.73.208]:25
posttls-finger: < 220-mail-cbf.charite.de ESMTP Postfix
posttls-finger: < 220 mail-cbf.charite.de ESMTP Postfix
posttls-finger: > EHLO amnesiac
posttls-finger: < 250-mail-cbf.charite.de
posttls-finger: < 250-SIZE 36700160
posttls-finger: < 250-ETRN
posttls-finger: < 250-STARTTLS
posttls-finger: < 250-ENHANCEDSTATUSCODES
posttls-finger: < 250 8BITMIME
posttls-finger: > STARTTLS
posttls-finger: < 220 2.0.0 Ready to start TLS
[...]
posttls-finger: Untrusted TLS connection established to mail-cbf-ext.charite.de[193.175.73.208]:25: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
posttls-finger: > EHLO amnesiac
posttls-finger: < 250-mail-cbf.charite.de
posttls-finger: < 250-SIZE 36700160
posttls-finger: < 250-ETRN
posttls-finger: < 250-ENHANCEDSTATUSCODES
posttls-finger: < 250 8BITMIME
posttls-finger: > QUIT
posttls-finger: < 221 2.0.0 Bye

The postscreen source code reads in part:

  static int psc_bdat_cmd(PSC_STATE *state, char *args)
  {
    const char *myname = "psc_bdat_cmd";

    /*
     * smtpd(8) incompatibility: postscreen(8) drops the connection, instead
     * of reading the entire BDAT chunk and staying in sync with the client.
     * Justification: postscreen(8) should never see BDAT from a legitimate
     * client, because 1) the server rejects every recipient, and 2) the
     * server does not announce PIPELINING.
     */
     ...

So I would expect that the BDAT command should not have been sent by the
client.  Given that it was sent, Postfix should arguably not report the
subsequent message a pipelining violation, but surely the BDAT command
itself is a violation of the same kind...

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: Postscreen vs. BDAT

Wietse Venema
In reply to this post by Ralf Hildebrandt-2
Ralf Hildebrandt:

> Today a fellow postmaster (using Exim) called me, they were having problems sending
> mail to charite.de. In my log I found:
>
> Sep  3 00:31:18 mail-cbf postfix/postscreen[34943]: CONNECT from [31.7.179.105]:38256 to [193.175.73.208]:25
> Sep  3 00:31:24 mail-cbf postfix/tlsproxy[39995]: CONNECT from [31.7.179.105]:38256
> Sep  3 00:31:24 mail-cbf postfix/tlsproxy[39995]: Anonymous TLS connection established from [31.7.179.105]:38256: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
> Sep  3 00:31:24 mail-cbf postfix/postscreen[34943]: NOQUEUE: reject: RCPT from [31.7.179.105]:38256: 450 4.3.2 Service currently unavailable; from=<[hidden email]>, to=<[hidden email]>, proto=ESMTP, helo=<smtpgw03.nextwerk.de>
> Sep  3 00:31:24 mail-cbf postfix/postscreen[34943]: COMMAND PIPELINING from [31.7.179.105]:38256 after BDAT: Received: from [213.182.136.75] (helo=NWH-EX13.nwcloud.loc)\r\n\tby smtpgw03.nextwerk.de with esmtpsa (
>
> After I disabled chunking, mail would flow again.
> Is this an Exim issue or a flaw in the newly introduced BDAT feature?

The client is at fault. Postscreen DOES NOT announce PIPELNING support.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Postscreen vs. BDAT

Ralf Hildebrandt-2
* Wietse Venema <[hidden email]>:

> Ralf Hildebrandt:
> > Today a fellow postmaster (using Exim) called me, they were having problems sending
> > mail to charite.de. In my log I found:
> >
> > Sep  3 00:31:18 mail-cbf postfix/postscreen[34943]: CONNECT from [31.7.179.105]:38256 to [193.175.73.208]:25
> > Sep  3 00:31:24 mail-cbf postfix/tlsproxy[39995]: CONNECT from [31.7.179.105]:38256
> > Sep  3 00:31:24 mail-cbf postfix/tlsproxy[39995]: Anonymous TLS connection established from [31.7.179.105]:38256: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
> > Sep  3 00:31:24 mail-cbf postfix/postscreen[34943]: NOQUEUE: reject: RCPT from [31.7.179.105]:38256: 450 4.3.2 Service currently unavailable; from=<[hidden email]>, to=<[hidden email]>, proto=ESMTP, helo=<smtpgw03.nextwerk.de>
> > Sep  3 00:31:24 mail-cbf postfix/postscreen[34943]: COMMAND PIPELINING from [31.7.179.105]:38256 after BDAT: Received: from [213.182.136.75] (helo=NWH-EX13.nwcloud.loc)\r\n\tby smtpgw03.nextwerk.de with esmtpsa (
> >
> > After I disabled chunking, mail would flow again.
> > Is this an Exim issue or a flaw in the newly introduced BDAT feature?
>
> The client is at fault. Postscreen DOES NOT announce PIPELNING support.

Thanks!

--
[*] sys4 AG

https://sys4.de, +49 (89) 30 90 46 64
Schleißheimer Straße 26/MG, 80333 München
                                           
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
Reply | Threaded
Open this post in threaded view
|

Re: Postscreen vs. BDAT

Wietse Venema
In reply to this post by Viktor Dukhovni
Viktor Dukhovni:

>     /*
>      * smtpd(8) incompatibility: postscreen(8) drops the connection, instead
>      * of reading the entire BDAT chunk and staying in sync with the client.
>      * Justification: postscreen(8) should never see BDAT from a legitimate
>      * client, because 1) the server rejects every recipient, and 2) the
>      * server does not announce PIPELINING.
>      */
>      ...
>
> So I would expect that the BDAT command should not have been sent by the
> client.  Given that it was sent, Postfix should arguably not report the
> subsequent message a pipelining violation, but surely the BDAT command
> itself is a violation of the same kind...

I have difficuty parsing your suggestion that postsccreen should
not consider this a pipelinig violation.

The client pipelines commands, postscreen does not announce
pipelining, therefore the client is at fault.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Postscreen vs. BDAT

Viktor Dukhovni


> On Sep 3, 2018, at 9:31 AM, Wietse Venema <[hidden email]> wrote:
>
> I have difficuty parsing your suggestion that postsccreen should
> not consider this a pipelinig violation.
>
> The client pipelines commands, postscreen does not announce
> pipelining, therefore the client is at fault.

Perhaps I was too cryptic, my comment was that the pipelining
violation was sending BDAT in the first place, while Ralf's
logs indicate that the pipelining violation was the message
payload after BDAT, which is not in itself a pipeling issue
since BDAT payload succeeds the command without a response
from the server.

--
--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: Postscreen vs. BDAT

Wietse Venema
Viktor Dukhovni:

>
>
> > On Sep 3, 2018, at 9:31 AM, Wietse Venema <[hidden email]> wrote:
> >
> > I have difficuty parsing your suggestion that postsccreen should
> > not consider this a pipelinig violation.
> >
> > The client pipelines commands, postscreen does not announce
> > pipelining, therefore the client is at fault.
>
> Perhaps I was too cryptic, my comment was that the pipelining
> violation was sending BDAT in the first place, while Ralf's
> logs indicate that the pipelining violation was the message
> payload after BDAT, which is not in itself a pipeling issue
> since BDAT payload succeeds the command without a response
> from the server.

OK, so the we have two pipelining claims.

1) postscreen's claim that something was pipelined after BDAT. That
claim is wrong, because it looks for the FIRST bytes after a BDAT
command. I can fix that.

    (a correct pipelining check would look for bytes pipelined after
    the LAST bytes of BDAT payload, but that requires processing
    the entire BDAT payload and is practical only with small payloads).

2) Exim sends BDAT after RCPT is rejected. That is a protocol error.
There is no pipelining error, because Exim did not pipeline BDAT
after RCPT. postscreen did not log "command pipelining after RCPT".

Sep  3 00:31:24 mail-cbf postfix/postscreen[34943]: NOQUEUE: reject: RCPT from [31.7.179.105]:38256: 450 4.3.2 Service currently unavailable; from=<[hidden email]>, to=<[hidden email]>, proto=ESMTP, helo=<smtpgw03.nextwerk.de>

Sep  3 00:31:24 mail-cbf postfix/postscreen[34943]: COMMAND PIPELINING from [31.7.179.105]:38256 after BDAT: Received: from [213.182.136.75] (helo=NWH-EX13.nwcloud.loc)\r\n\tby smtpgw03.nextwerk.de with esmtpsa (

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Postscreen vs. BDAT

Viktor Dukhovni


> On Sep 3, 2018, at 10:45 AM, Wietse Venema <[hidden email]> wrote:
>
> 2) Exim sends BDAT after RCPT is rejected. That is a protocol error.
> There is no pipelining error, because Exim did not pipeline BDAT
> after RCPT. postscreen did not log "command pipelining after RCPT".

It is perhaps possible that Exim sent BDAT separately from RCPT
with a consequent Nagle delay, so that Postfix saw just RCPT TO,
but BDAT was sent before Exim saw (or bothered to read the response
for) the rejected recipient.

It is also possible that the Exim version in question is out of date,
I recall seeing various bug reports on the Exim-users list about the
CHUNKING support in Exim, even some security issues.  Don't know whether
the same symptoms are to be expected from a fully-patched version.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: Postscreen vs. BDAT

Ralf Hildebrandt-2
> It is also possible that the Exim version in question is out of date,
> I recall seeing various bug reports on the Exim-users list about the
> CHUNKING support in Exim, even some security issues.  Don't know whether
> the same symptoms are to be expected from a fully-patched version.

According to the headers smtpgw03.nextwerk.de is running Exim 4.90_1
--
[*] sys4 AG

https://sys4.de, +49 (89) 30 90 46 64
Schleißheimer Straße 26/MG, 80333 München
                                           
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
Reply | Threaded
Open this post in threaded view
|

PATCH: Postscreen vs. BDAT

Wietse Venema
Ralf Hildebrandt:
> > It is also possible that the Exim version in question is out of date,
> > I recall seeing various bug reports on the Exim-users list about the
> > CHUNKING support in Exim, even some security issues.  Don't know whether
> > the same symptoms are to be expected from a fully-patched version.
>
> According to the headers smtpgw03.nextwerk.de is running Exim 4.90_1

Also released as part of postfix-3.4-20180903.

        Wietse

diff --exclude=man --exclude=html --exclude=README_FILES --exclude=INSTALL --exclude=.indent.pro --exclude=Makefile.in -r -ur /var/tmp/postfix-3.4-20180827/src/postscreen/postscreen_smtpd.c ./src/postscreen/postscreen_smtpd.c
--- /var/tmp/postfix-3.4-20180827/src/postscreen/postscreen_smtpd.c 2018-08-25 19:36:13.000000000 -0400
+++ ./src/postscreen/postscreen_smtpd.c 2018-09-03 18:49:28.000000000 -0400
@@ -737,6 +737,7 @@
 #define PSC_SMTPD_CMD_FLAG_DESTROY (1<<1) /* dangling pointer alert */
 #define PSC_SMTPD_CMD_FLAG_PRE_TLS (1<<2) /* allowed with mandatory TLS */
 #define PSC_SMTPD_CMD_FLAG_SUSPEND (1<<3) /* suspend command engine */
+#define PSC_SMTPD_CMD_FLAG_HAS_PAYLOAD (1<<4) /* command has payload */
 
 static const PSC_SMTPD_COMMAND command_table[] = {
     "HELO", psc_helo_cmd, PSC_SMTPD_CMD_FLAG_ENABLE | PSC_SMTPD_CMD_FLAG_PRE_TLS,
@@ -749,7 +750,7 @@
     "RCPT", psc_rcpt_cmd, PSC_SMTPD_CMD_FLAG_ENABLE,
     "DATA", psc_data_cmd, PSC_SMTPD_CMD_FLAG_ENABLE | PSC_SMTPD_CMD_FLAG_DESTROY,
     /* ".", psc_dot_cmd, PSC_SMTPD_CMD_FLAG_NONE, */
-    "BDAT", psc_bdat_cmd, PSC_SMTPD_CMD_FLAG_ENABLE | PSC_SMTPD_CMD_FLAG_DESTROY,
+    "BDAT", psc_bdat_cmd, PSC_SMTPD_CMD_FLAG_ENABLE | PSC_SMTPD_CMD_FLAG_DESTROY | PSC_SMTPD_CMD_FLAG_HAS_PAYLOAD,
     "RSET", psc_rset_cmd, PSC_SMTPD_CMD_FLAG_ENABLE,
     "NOOP", psc_noop_cmd, PSC_SMTPD_CMD_FLAG_ENABLE | PSC_SMTPD_CMD_FLAG_PRE_TLS,
     "VRFY", psc_vrfy_cmd, PSC_SMTPD_CMD_FLAG_ENABLE,
@@ -1031,7 +1032,8 @@
     }
  }
  /* Command PIPELINING test. */
- if ((state->flags & PSC_STATE_MASK_PIPEL_TODO_SKIP)
+ if ((state->flags & PSC_SMTPD_CMD_FLAG_HAS_PAYLOAD) == 0
+    && (state->flags & PSC_STATE_MASK_PIPEL_TODO_SKIP)
     == PSC_STATE_FLAG_PIPEL_TODO && !PSC_SMTPD_BUFFER_EMPTY(state)) {
     printable(command, '?');
     PSC_SMTPD_ESCAPE_TEXT(psc_temp, PSC_SMTPD_PEEK_DATA(state),