patch: smpd insert DSN request

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

patch: smpd insert DSN request

A. Schulze

Hello,

DSN for successfully deliveries are not enabled by default. They have  
to be requested
by an smtp client using extra parameter on RFC5321.MAILFROM an RFC5321.RCPTTO

But not every MUA has the ability to do this. Some years ago I had a  
requirement to
force DSN on successfully deliveries for every message send by a  
simple smtp client as a dedicated mailstream.

So I wrote a little patch to let postfix smtpd insert the necessary data.
The function must be enabled by an new option. Now other people told  
me they find that feature useful too.
So I like to publish that patch. Maybe it could be included in postfix  
some day.

usage: master.cf
submission inet  n - n - - smtpd
  -o syslog_name=postfix/submission_with_dsn
  -o smtpd_force_dsn_on_success=yes


Andreas


smtpd_force_dsn_on_success.patch (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: patch: smpd insert DSN request

Wietse Venema
Andreas Schulze:

>
> Hello,
>
> DSN for successfully deliveries are not enabled by default. They have  
> to be requested
> by an smtp client using extra parameter on RFC5321.MAILFROM an RFC5321.RCPTTO
>
> But not every MUA has the ability to do this. Some years ago I had a  
> requirement to
> force DSN on successfully deliveries for every message send by a  
> simple smtp client as a dedicated mailstream.
>
> So I wrote a little patch to let postfix smtpd insert the necessary data.

There is no need to patch. Use smtpd_command_filter instead. See the
examples in the postconf(5) manpage for how to modify MAIL FROM,
RCPT TO, and so on.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: patch: smpd insert DSN request

Wietse Venema
Wietse Venema:

> Andreas Schulze:
> >
> > Hello,
> >
> > DSN for successfully deliveries are not enabled by default. They have  
> > to be requested
> > by an smtp client using extra parameter on RFC5321.MAILFROM an RFC5321.RCPTTO
> >
> > But not every MUA has the ability to do this. Some years ago I had a  
> > requirement to
> > force DSN on successfully deliveries for every message send by a  
> > simple smtp client as a dedicated mailstream.
> >
> > So I wrote a little patch to let postfix smtpd insert the necessary data.
>
> There is no need to patch. Use smtpd_command_filter instead. See the
> examples in the postconf(5) manpage for how to modify MAIL FROM,
> RCPT TO, and so on.

Something like this (uses PCRE syntax) should do the job. The
first rule excludes clients that send NOTIFY options, and the
second adds forced NOTIFY for clients that don't.

/^(RCPT\s+TO:<.*>.*\s+NOTIFY=.*)/ $1
/^(RCPT\s+TO:<.*>.*)/ $1 NOTIFY=SUCCESS,DELAY,FAILURE

There is a problem with your patch: it turns off forced DSN for all
SMTP clients that are serviced by the same SMTP daemon process.

        Wietse

Reply | Threaded
Open this post in threaded view
|

Re: patch: smpd insert DSN request

A. Schulze
Wietse Venema:
> There is a problem with your patch: it turns off forced DSN for all
> SMTP clients that are serviced by the same SMTP daemon process.

Oh, yes, now I see it too. var_force_dsn_on_success is global per process?
My usecases never trigger that situation.

One point more to use your suggested solution.

Andreas
Reply | Threaded
Open this post in threaded view
|

Re: patch: smpd insert DSN request (Update)

A. Schulze
In reply to this post by Wietse Venema

wietse:
> /^(RCPT\s+TO:<.*>.*\s+NOTIFY=.*)/ $1
> /^(RCPT\s+TO:<.*>.*)/ $1 NOTIFY=SUCCESS,DELAY,FAILURE

the regex above don't match on 'RCPT TO: <[hidden email]>' ( SPACE  
after colon )

I use now:
/^(RCPT\s+TO:\s*<.*>.*\s+NOTIFY=.*)/ $1
/^(RCPT\s+TO:\s*<.*>.*)/ $1 NOTIFY=SUCCESS,DELAY,FAILURE

Andreas

Reply | Threaded
Open this post in threaded view
|

Re: patch: smpd insert DSN request (Update)

Viktor Dukhovni
On Wed, Nov 12, 2014 at 05:56:38PM +0100, A. Schulze wrote:
>
> wietse:
> >/^(RCPT\s+TO:<.*>.*\s+NOTIFY=.*)/ $1
> >/^(RCPT\s+TO:<.*>.*)/ $1 NOTIFY=SUCCESS,DELAY,FAILURE
>
> the regex above don't match on 'RCPT TO: <[hidden email]>' ( SPACE after
> colon )

Note the extra space violates RFCs 5321, 2821, 821.  However, it
was historically accepted by Sendmail, and is accepted also by
Postfix as a compatibility crutch.

> I use now:
> /^(RCPT\s+TO:\s*<.*>.*\s+NOTIFY=.*)/ $1
> /^(RCPT\s+TO:\s*<.*>.*)/ $1 NOTIFY=SUCCESS,DELAY,FAILURE

So a cleaner approach might be:

    /^RCPT\s+TO:\s*(<.*?>.*?)\s+(NOTIFY=.*)/ RCPT TO:$1 $2
    /^RCPT\s+TO:\s*(<.*>.*)/ RCPT TO:$1 NOTIFY=SUCCESS,DELAY,FAILURE

[ Since vertical tabs, and other notional PCRE white-space characters
are not SMTP white-space, PCRE whitespace matched with "\s+" is
canonicalized to a space. ]

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: patch: smpd insert DSN request (Update)

Wietse Venema
Viktor Dukhovni:

> On Wed, Nov 12, 2014 at 05:56:38PM +0100, A. Schulze wrote:
> >
> > wietse:
> > >/^(RCPT\s+TO:<.*>.*\s+NOTIFY=.*)/ $1
> > >/^(RCPT\s+TO:<.*>.*)/ $1 NOTIFY=SUCCESS,DELAY,FAILURE
> >
> > the regex above don't match on 'RCPT TO: <[hidden email]>' ( SPACE after
> > colon )
>
> Note the extra space violates RFCs 5321, 2821, 821.  However, it
> was historically accepted by Sendmail, and is accepted also by
> Postfix as a compatibility crutch.
>
> > I use now:
> > /^(RCPT\s+TO:\s*<.*>.*\s+NOTIFY=.*)/ $1
> > /^(RCPT\s+TO:\s*<.*>.*)/ $1 NOTIFY=SUCCESS,DELAY,FAILURE
>
> So a cleaner approach might be:
>
>     /^RCPT\s+TO:\s*(<.*?>.*?)\s+(NOTIFY=.*)/ RCPT TO:$1 $2
>     /^RCPT\s+TO:\s*(<.*>.*)/ RCPT TO:$1 NOTIFY=SUCCESS,DELAY,FAILURE
>
> [ Since vertical tabs, and other notional PCRE white-space characters
> are not SMTP white-space, PCRE whitespace matched with "\s+" is
> canonicalized to a space. ]

I suppose that "clean" is in the eye of the beholder.  I don't think
that we have to perpetuate workarounds for bugs that were already
uncommon 20 years ago.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: patch: smpd insert DSN request (Update)

Wietse Venema
In reply to this post by A. Schulze
A. Schulze:
>
> wietse:
> > /^(RCPT\s+TO:<.*>.*\s+NOTIFY=.*)/ $1
> > /^(RCPT\s+TO:<.*>.*)/ $1 NOTIFY=SUCCESS,DELAY,FAILURE
>
> the regex above don't match on 'RCPT TO: <[hidden email]>' ( SPACE  
> after colon )

That is invalid syntax.  What software (other than home-grown
scripts) sends commands like that?

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: invalid SMTP syntax

Claus Assmann-22
On Wed, Nov 12, 2014, Wietse Venema wrote:
> A. Schulze:

> > the regex above don't match on 'RCPT TO: <[hidden email]>' ( SPACE  

> That is invalid syntax.  What software (other than home-grown
> scripts) sends commands like that?

Last time I looked: at least cashedge.com, used by several US
banks(?) :-(
Probably some Java stuff.

{\begin rant}
I sent patches to various open source software projects about
this, all of them have been fixed, but I couldn't find anyone
at cashedge... I filed a case with my bank years ago, they
told me it works fine with "all big providers" and I should
get an e-mail account (for my financial information!) there.
{\end rant}
Reply | Threaded
Open this post in threaded view
|

Re: invalid SMTP syntax

Wietse Venema
Claus Assmann:

> On Wed, Nov 12, 2014, Wietse Venema wrote:
> > A. Schulze:
>
> > > the regex above don't match on 'RCPT TO: <[hidden email]>' ( SPACE  
>
> > That is invalid syntax.  What software (other than home-grown
> > scripts) sends commands like that?
>
> Last time I looked: at least cashedge.com, used by several US
> banks(?) :-(
> Probably some Java stuff.
>
> {\begin rant}
> I sent patches to various open source software projects about
> this, all of them have been fixed, but I couldn't find anyone
> at cashedge... I filed a case with my bank years ago, they
> told me it works fine with "all big providers" and I should
> get an e-mail account (for my financial information!) there.
> {\end rant}

Thanks. I'll update the manpage examples. We don't want people
to lose mail from their bank.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: patch: smpd insert DSN request (Update)

A. Schulze
In reply to this post by Wietse Venema

wietse:

> A. Schulze:
>>
>> wietse:
>>> /^(RCPT\s+TO:<.*>.*\s+NOTIFY=.*)/ $1
>>> /^(RCPT\s+TO:<.*>.*)/ $1 NOTIFY=SUCCESS,DELAY,FAILURE
>>
>> the regex above don't match on 'RCPT TO: <[hidden email]>' ( SPACE
>> after colon )
>
> That is invalid syntax.  What software (other than home-grown  
> scripts) sends commands like that?
Android-4.4.2 Mail

Nov 12 17:45:33 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 220 mail.example.org ESMTP
Nov 12 17:45:33 mail.example.org postfix/smtpd[17904]: <  
unknown[192.0.2.25]: EHLO [192.0.2.25]
Nov 12 17:45:33 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 250-mail.example.org
Nov 12 17:45:33 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 250-PIPELINING
Nov 12 17:45:33 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 250-SIZE 40000000
Nov 12 17:45:33 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 250-STARTTLS
Nov 12 17:45:33 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 250-ENHANCEDSTATUSCODES
Nov 12 17:45:33 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 250-8BITMIME
Nov 12 17:45:33 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 250-DSN
Nov 12 17:45:33 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 250 SMTPUTF8
Nov 12 17:45:34 mail.example.org postfix/smtpd[17904]: <  
unknown[192.0.2.25]: STARTTLS
Nov 12 17:45:34 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 220 2.0.0 Ready to start TLS
Nov 12 17:45:34 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 250-mail.example.org
Nov 12 17:45:34 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 250-PIPELINING
Nov 12 17:45:34 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 250-SIZE 0815
Nov 12 17:45:34 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 250-AUTH PLAIN
Nov 12 17:45:34 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 250-ENHANCEDSTATUSCODES
Nov 12 17:45:34 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 250-8BITMIME
Nov 12 17:45:34 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 250-DSN
Nov 12 17:45:34 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 250 SMTPUTF8
Nov 12 17:45:34 mail.example.org postfix/smtpd[17904]: <  
unknown[192.0.2.25]: AUTH PLAIN *****
Nov 12 17:45:34 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 235 2.7.0 Authentication successful
Nov 12 17:45:34 mail.example.org postfix/smtpd[17904]: <  
unknown[192.0.2.25]: RSET
Nov 12 17:45:34 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 250 2.0.0 Ok
Nov 12 17:45:34 mail.example.org postfix/smtpd[17904]: <  
unknown[192.0.2.25]: MAIL FROM: <sender@local.****>
Nov 12 17:45:34 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 250 2.1.0 Ok
Nov 12 17:45:34 mail.example.org postfix/smtpd[17904]: <  
unknown[192.0.2.25]: RCPT TO: <receiver@external.*****>
Nov 12 17:45:34 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 250 2.1.5 Ok
Nov 12 17:45:34 mail.example.org postfix/smtpd[17904]: <  
unknown[192.0.2.25]: DATA
Nov 12 17:45:34 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 354 End data with <CR><LF>.<CR><LF>
Nov 12 17:45:35 mail.example.org postfix/cleanup[17907]:  
3jdBdV53xYzDfk: message-id=<****@email.android.com>
Nov 12 17:45:35 mail.example.org postfix/qmgr[17839]: 3jdBdV53xYzDfk:  
from=<sender@local.****>, size=****, nrcpt=1 (queue active)
Nov 12 17:45:35 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 250 2.0.0 Ok: queued as 3jdBdV53xYzDfk
Nov 12 17:50:35 mail.example.org postfix/smtpd[17904]: >  
unknown[192.0.2.25]: 421 4.4.2 mail.example.org Error: timeout  
exceeded, servertime=Nov 12 17:50:35, client=192.0.2.25


funny to note:
once android mail has a tcp session esteblished it hold the session  
open until timeout



Reply | Threaded
Open this post in threaded view
|

Re: invalid SMTP syntax: Android-4.4.2 Mail

Claus Assmann-22
On Fri, Nov 14, 2014, A. Schulze wrote:

> Android-4.4.2 Mail

If you have a chance, can you contact the developers and ask
them to fix it?

> unknown[192.0.2.25]: MAIL FROM: <sender@local.****>
> unknown[192.0.2.25]: RCPT TO: <receiver@external.*****>

RFC 5321 explicitly lists this error (for those who have trouble
understanding RFC 821 or RFC 2821):

   Since it has been a common source of errors, it is worth noting that
   spaces are not permitted on either side of the colon following FROM
   in the MAIL command or TO in the RCPT command.  The syntax is exactly
   as given above.