Feature Request: lmtp --> content_filter --> lmtpd

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Feature Request: lmtp --> content_filter --> lmtpd

Rick van Rein
Hello,

Postfix normally filters mail using a pipeline like

smtp --> content_filter --> smtpd

but it lacks the lmtpd that would also enable

lmtp --> content_filter --> lmtpd


Why is that useful?

I've seen a few questions posted about forking mail.  This is usually a
bad idea for incoming mail, but when it is locally generated there can
be reasons :- For reasons of privacy and email management, I am trying
to mask the sender's address into an externally visible alias, but which
alias makes sense would depend on the recipient.  No problem when
there's one recipient, but a problem when there are many; it ends up
forking the email into groups of recipients, dependent on which address
they may get to see.

When forking the email by sending it to multiple output processes, the
most dreadful half-way failure conditions can occur, which degrade the
transactional quality level of email handling.  With LMTP in and out
however, a content_filter can easily return which of the recipients have
gotten their email sent, and which did not.  Postfix would then return a
report based on what it got replied into the initiating lmtp process.


Would this be considered a bad idea, or just a new idea?


Thanks,

Rick van Rein
Reply | Threaded
Open this post in threaded view
|

Re: Feature Request: lmtp --> content_filter --> lmtpd

Viktor Dukhovni
On Tue, Aug 15, 2017 at 10:25:19PM +0200, Rick van Rein wrote:

> Postfix normally filters mail using a pipeline like
>
> smtp --> content_filter --> smtpd

The SMTP server atomically commits a single queue-file, so there's
no advantage to talking LMTP.


> but it lacks the lmtpd that would also enable
>
> lmtp --> content_filter --> lmtpd

Actually, you don't need LMTP on the second leg of this diagram.
Just have your content_filter translate between LMTP and SMTP.
Each envelope written by the content filter is still a single queue
file, so using LMTP on the back-end link just complicates the
protocol.

> When forking the email by sending it to multiple output processes, the
> most dreadful half-way failure conditions can occur, which degrade the
> transactional quality level of email handling.  With LMTP in and out
> however, a content_filter can easily return which of the recipients have
> gotten their email sent, and which did not.  Postfix would then return a
> report based on what it got replied into the initiating lmtp process.
>
>
> Would this be considered a bad idea, or just a new idea?

By all means deploy an LMTP content_filter, but use SMTP to re-inject
the filtered messages.  If a group of recipients temp-fails the
re-injection, send a failure code for that group to the front-end
LMTP client.

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Feature Request: lmtp --> content_filter --> lmtpd

Rick van Rein
Hi Viktor,

> By all means deploy an LMTP content_filter, but use SMTP to re-inject
> the filtered messages.  If a group of recipients temp-fails the
> re-injection, send a failure code for that group to the front-end
> LMTP client.

Yes, that should also work, thanks.

I was focussing on passing feedback for individual recipients in a
one-by-one fashion from lmtpd back to lmtp, but it is actually even
nicer to do it per group -- so that a resend can be tried for the
group and they get to see each other's addresses to share.

Thanks!
 -Rick
Reply | Threaded
Open this post in threaded view
|

Re: Feature Request: lmtp --> content_filter --> lmtpd

Viktor Dukhovni

> On Aug 16, 2017, at 1:53 AM, Rick van Rein <[hidden email]> wrote:
>
> I was focussing on passing feedback for individual recipients in a
> one-by-one fashion from lmtpd back to lmtp, but it is actually even
> nicer to do it per group -- so that a resend can be tried for the
> group and they get to see each other's addresses to share.

The above does not make much sense to me.  Whatever the batching
of recipients into SMTP envelopes (possibly down to "singleton"
messages per recipient) each such message may as well be re-injected
via SMTP, and its response "stuttered" per envelope recipient back
to the upstream LMTP client.  You will of course need to be careful
to maintain the proper order of "." responses to match the order of
"2XX" "RCPT TO" responses in the upstream envelope.

You content filter should open multiple SMTP sessions in parallel
and clone as much of the message body as possible to each one
round-robin.  The collect all the "." responses in parallel,
associate each one with the right list of upstream recipients,
then transmit all the per-recipient replies upstream in the
right order.

Since the re-injection Postfix service just writes a queue
file and does not split envelopes (that happens later when
the queue manager passes recipient batches to delivery
agents) there's no option of that service giving usefully
different per-recipient replies at ".".

So, while an "lmtpd" in Postfix would not be difficult
to implement, it would not be useful.

--
        Viktor.