Bounce mails manually

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Bounce mails manually

Peer Heinlein

I'm thinking about a way how to bounce mails manually *without* setting
up a transport-map. Just bei CLI by the admin for a given Queue-ID.

I'd love having a postsuper-commando to move a mail into "the bounce
queue". Is something like that possible?

Peer



--
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin

http://www.heinlein-support.de

Tel: 030 / 405051-42
Fax: 030 / 405051-19

Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht
Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin


signature.asc (501 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Bounce mails manually

A. Schulze

Peer Heinlein:

> I'm thinking about a way how to bounce mails manually *without* setting
> up a transport-map. Just bei CLI by the admin for a given Queue-ID.
>
> I'd love having a postsuper-commando to move a mail into "the bounce
> queue". Is something like that possible?

thanks for the question.
I also needed such feature some times.

# postbounce <queue-id>

Andreas

Reply | Threaded
Open this post in threaded view
|

Re: Bounce mails manually

Birta Levente
In reply to this post by Peer Heinlein
On 25/02/2014 10:10, Peer Heinlein wrote:
> I'm thinking about a way how to bounce mails manually *without* setting
> up a transport-map. Just bei CLI by the admin for a given Queue-ID.
>
> I'd love having a postsuper-commando to move a mail into "the bounce
> queue". Is something like that possible?

There's no bounce queue.

Why not just delete from the queue?

--
            Levi

Reply | Threaded
Open this post in threaded view
|

Re: Bounce mails manually

A. Schulze

Birta Levente:

> Why not just delete from the queue?

from senders perspective that message is lost.
sometimes it's useful to clear bounce back to sender.

Andreas

Reply | Threaded
Open this post in threaded view
|

Re: Bounce mails manually

Birta Levente
On 25/02/2014 10:49, Andreas Schulze wrote:
>
> Birta Levente:
>
>> Why not just delete from the queue?
>
> from senders perspective that message is lost.
> sometimes it's useful to clear bounce back to sender.
>
Yes, but you sould give some reason why is bounced ... which IMHO is
something permanent ... so you just set up one time some map and no more
care about that problem.

--
            Levi

Reply | Threaded
Open this post in threaded view
|

Re: Bounce mails manually

A. Schulze

Birta Levente:

> Yes, but you sould give some reason why is bounced ... which IMHO is  
> something permanent ...
good point!
# postbounce <queue-id> <reason>

> so you just set up one time some map and no more care about that problem.
just this is unwanted and the reason for the request.

Andreas

Reply | Threaded
Open this post in threaded view
|

Re: Bounce mails manually

Birta Levente
On 25/02/2014 11:02, Andreas Schulze wrote:

>
> Birta Levente:
>
>> Yes, but you sould give some reason why is bounced ... which IMHO is
>> something permanent ...
> good point!
> # postbounce <queue-id> <reason>
>
>> so you just set up one time some map and no more care about that
>> problem.
> just this is unwanted and the reason for the request.
>

Thats unimaginable for me: only one sender and only one time make that
mistake.

--
            Levi

Reply | Threaded
Open this post in threaded view
|

Re: Bounce mails manually

lists@rhsoft.net


Am 25.02.2014 10:09, schrieb Birta Levente:

> On 25/02/2014 11:02, Andreas Schulze wrote:
>>
>> Birta Levente:
>>
>>> Yes, but you sould give some reason why is bounced ... which IMHO is something permanent ...
>> good point!
>> # postbounce <queue-id> <reason>
>>
>>> so you just set up one time some map and no more care about that problem.
>> just this is unwanted and the reason for the request.
>>
>
> Thats unimaginable for me: only one sender and only one time make that mistake

if i write for every random mistake a user out of 500 makes
a entry in a map postfix would epxlode - the idea behind
"bounce with reason" is to

* get rid of that message
* explain the user what he did wrong
Reply | Threaded
Open this post in threaded view
|

Re: Bounce mails manually

Wietse Venema
In reply to this post by Peer Heinlein
Peer Heinlein:
> I'm thinking about a way how to bounce mails manually *without* setting
> up a transport-map. Just bei CLI by the admin for a given Queue-ID.
>
> I'd love having a postsuper-commando to move a mail into "the bounce
> queue". Is something like that possible?

That would require infrastructure that does not exist at this time.
And in order to be part of Postfix it would have to be useful for
more than one person.

I don't know what people are asking for:

1 - Bounce all recipients of one specific queue file? That can be
    done with a small program.

2 - Bouncing only specific recipients is more work, and it requires
    a different program than the one in the previous paragraph which
    bounces all recipients.

Case 1 can be augmented with a script that parses mailq output and
that makes a selection based on sender, recipient or other attibute.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Bounce mails manually

A. Schulze

wietse:

> I don't know what people are asking for:
>   1 - Bounce all recipients of one specific queue file
>   2 - Bouncing only specific recipients

option 1 (for me)

in case of trouble I do
  - mailq for visual overview
  - pfqgrep -r/-s address -i | postsuper -d -

In this context it would sometimes be useful to
postbounce <queue-id> <reason>

Andreas


Reply | Threaded
Open this post in threaded view
|

Re: Bounce mails manually

Noel Jones-2
On 2/26/2014 12:41 AM, Andreas Schulze wrote:

>
> wietse:
>
>> I don't know what people are asking for:
>>   1 - Bounce all recipients of one specific queue file
>>   2 - Bouncing only specific recipients
>
> option 1 (for me)
>
> in case of trouble I do
>  - mailq for visual overview
>  - pfqgrep -r/-s address -i | postsuper -d -
>
> In this context it would sometimes be useful to
> postbounce <queue-id> <reason>
>
> Andreas

Same here, option 1. It would occasionally be useful to bounce all
undelivered recipients of a queue file. But this isn't something I'd
use every week.



  -- Noel Jones
Reply | Threaded
Open this post in threaded view
|

Re: Bounce mails manually

Wietse Venema
Noel Jones:

> On 2/26/2014 12:41 AM, Andreas Schulze wrote:
> > wietse:
> >
> >> I don't know what people are asking for:
> >>   1 - Bounce all recipients of one specific queue file
> >>   2 - Bouncing only specific recipients
> >
> > option 1 (for me)
> >
> > in case of trouble I do
> >  - mailq for visual overview
> >  - pfqgrep -r/-s address -i | postsuper -d -
> >
> > In this context it would sometimes be useful to
> > postbounce <queue-id> <reason>
> >
> > Andreas
>
> Same here, option 1. It would occasionally be useful to bounce all
> undelivered recipients of a queue file. But this isn't something I'd
> use every week.

This can in effect bounce only the remaining recipients. The other
ones (bounced or delivered) are already renamed to "done" and
therefore effectively hidden.

Note 1: this requires that Postfix daemons are available.  If no
bounce message can be queued, then mail will not be bounced
and will stay queued.

Note 2: just like "postsuper -d" this may hit the wrong message when
the system is configured for short queue IDs, as those may be reused.

Note 3: like the queue manager this needs to distinguish between
regular mail, "sendmail -bv", address verification probes, VERP,
and so on. Only a few of these types are supposed to be returned
to the sender; the other types require different handling.

Note 4: like the queue manager this needs to lock a queue file
before starting work on it, mark a bounced recipient as "done", and
clean up defer logs, trace logs etc.  before removing the queue
file.

There has to be a better way that avoids massive code duplication
and the resulting maintenance hell.

One possibility is to introduce a new one-byte flag in Postfix queue
files that is set when the email message must "bounce now", and to
introduce a new queue subdirectory with text files (same names as
message queue files) that say why the corresponding email message
is to be bounced.  This can then be processed with the existing
queue manager after minimal modification.

With this, part of the work happens in "postsuper -b" (write the
reason to a text file in the new queue subdirectory, then flip the
message queue file bit that says "bounce this message"). The other
parts happen in the queue manager (read the new flag byte and pass
it on in some simplified bounce request, and in the bounce daemon
(read the bounce reason from the text file in the new queue
subdirectory).

This is less work, because it extends code that already works,
without duplicating existing logic into another program.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Bounce mails manually

Noel Jones-2
On 2/26/2014 9:53 AM, Wietse Venema wrote:

> Noel Jones:
>> On 2/26/2014 12:41 AM, Andreas Schulze wrote:
>>> wietse:
>>>
>>>> I don't know what people are asking for:
>>>>   1 - Bounce all recipients of one specific queue file
>>>>   2 - Bouncing only specific recipients
>>>
>>> option 1 (for me)
>>>
>>> in case of trouble I do
>>>  - mailq for visual overview
>>>  - pfqgrep -r/-s address -i | postsuper -d -
>>>
>>> In this context it would sometimes be useful to
>>> postbounce <queue-id> <reason>
>>>
>>> Andreas
>>
>> Same here, option 1. It would occasionally be useful to bounce all
>> undelivered recipients of a queue file. But this isn't something I'd
>> use every week.
>
> This can in effect bounce only the remaining recipients. The other
> ones (bounced or delivered) are already renamed to "done" and
> therefore effectively hidden.
>
> Note 1: this requires that Postfix daemons are available.  If no
> bounce message can be queued, then mail will not be bounced
> and will stay queued.
>
> Note 2: just like "postsuper -d" this may hit the wrong message when
> the system is configured for short queue IDs, as those may be reused.
>
> Note 3: like the queue manager this needs to distinguish between
> regular mail, "sendmail -bv", address verification probes, VERP,
> and so on. Only a few of these types are supposed to be returned
> to the sender; the other types require different handling.
>
> Note 4: like the queue manager this needs to lock a queue file
> before starting work on it, mark a bounced recipient as "done", and
> clean up defer logs, trace logs etc.  before removing the queue
> file.
>
> There has to be a better way that avoids massive code duplication
> and the resulting maintenance hell.
>
> One possibility is to introduce a new one-byte flag in Postfix queue
> files that is set when the email message must "bounce now", and to
> introduce a new queue subdirectory with text files (same names as
> message queue files) that say why the corresponding email message
> is to be bounced.  This can then be processed with the existing
> queue manager after minimal modification.
>
> With this, part of the work happens in "postsuper -b" (write the
> reason to a text file in the new queue subdirectory, then flip the
> message queue file bit that says "bounce this message"). The other
> parts happen in the queue manager (read the new flag byte and pass
> it on in some simplified bounce request, and in the bounce daemon
> (read the bounce reason from the text file in the new queue
> subdirectory).
>
> This is less work, because it extends code that already works,
> without duplicating existing logic into another program.
>
> Wietse
>


I expect this doesn't work the way I think, but what about pointing
whatever the queue file uses for the content filter flag to the
bounce or error transport? Wouldn't that cause the message to bounce
on the next queue run without much new code?


  -- Noel Jones
Reply | Threaded
Open this post in threaded view
|

Re: Bounce mails manually

Viktor Dukhovni
On Wed, Feb 26, 2014 at 02:00:47PM -0600, Noel Jones wrote:

> I expect this doesn't work the way I think, but what about pointing
> whatever the queue file uses for the content filter flag to the
> bounce or error transport? Wouldn't that cause the message to bounce
> on the next queue run without much new code?

Queue files never grow after they are created.  All subsequent
modifications are atomic in-place updates of 1-byte recipient
recipient record type markers (from "recipient" to "done-recipient").

While the part of the intent of this design (not needing more space
to finish processing a message) is not always effective, with say
COW filesystems such as ZFS, it is a design feature that should
not be changed since the atomicity of the updates is also important.

It may be simpler to create a queue sub-directory, that is scanned
whenever the deferred queue is scanned, such that all messages
found there are automatically bounced with bounce messages per the
bounce logs for each message.

To bounce a message, the queue file is locked, bounce log entries
are written for each remaining recipient, and the message is moved
to the new queue.  No unsafe (non-atomic) changes are made to the
original queue file.

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Bounce mails manually

Wietse Venema
In reply to this post by Noel Jones-2
Noel Jones:
> I expect this doesn't work the way I think, but what about pointing
> whatever the queue file uses for the content filter flag to the
> bounce or error transport? Wouldn't that cause the message to bounce
> on the next queue run without much new code?

Indeed, not, because you can't change the queue file size.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Bounce mails manually

Noel Jones-2
On 2/26/2014 2:34 PM, Wietse Venema wrote:

> Noel Jones:
>> I expect this doesn't work the way I think, but what about pointing
>> whatever the queue file uses for the content filter flag to the
>> bounce or error transport? Wouldn't that cause the message to bounce
>> on the next queue run without much new code?
>
> Indeed, not, because you can't change the queue file size.
>
> Wietse
>

Ok, I see.  (and thanks Viktor for the in-depth explanation)




  -- Noel Jones
Reply | Threaded
Open this post in threaded view
|

Re: Bounce mails manually\

Wietse Venema
In reply to this post by Viktor Dukhovni
Viktor Dukhovni:
> To bounce a message, the queue file is locked, bounce log entries
> are written for each remaining recipient, and the message is moved
> to the new queue.  No unsafe (non-atomic) changes are made to the
> original queue file.

If this logfile update bypasses the queue manager or bounce daemon,
then 1) it suffers from race conditions, and 2) it results in the
massive code duplication that I must avoid.

First, you can't bypass the queue manager.  If your stand-alone
program doesn't coordinate with the queue manager, then the bounce
daemon will delete a bounce logfile while your stand-alone program
updates it. Besides you are forgetting to clean up orphaned defer
logs, trace logs, etc. More race conditions.

Second, you can't bypass the bounce client because of note 3.

    Note 3: like the queue manager this needs to distinguish between
    regular mail, "sendmail -bv", address verification probes, VERP,
    and so on. Only a few of these types are supposed to be returned
    to the sender; the other types require different handling.

But wait, there is more.  Just like the queue manager a stand-alone
program would need to precisely parse queue files such as parsing
the DSN records, the original-recipient records, queue file flags,
and more, so that it can create a proper bounce request (or whatever
is appropriate given the kind of message).  It's not sufficient to
do a half-assed parse like mailq/showq does.

Since we can't bypass the queue manager, bounce client and daemon,
and something has to parse queue files with approximately the
precision of the queue manager, it is better to leave those things
with the queue manager only and just set an atomic flag that this
file needs to be bounced. Writing the reason to a dedicated
subdirectory is not a problem.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Bounce mails manually\

A. Schulze

wietse:

> But wait, there is more....

does not sound like an easy job.
just an idea: if the timestamp of a queuefile is relevant, could a  
changed time
of a queuefile be interpreted as "bounce immediately" ?
for example timestamp to a fixed date near 1.1.1970

Andreas


Reply | Threaded
Open this post in threaded view
|

Re: Bounce mails manually\

Viktor Dukhovni
On Wed, Feb 26, 2014 at 10:14:10PM +0100, Andreas Schulze wrote:

> >But wait, there is more....
>
> does not sound like an easy job.

The difficult parts are not in marking the queue file.

> just an idea: if the timestamp of a queuefile is relevant, could a
> changed time
> of a queuefile be interpreted as "bounce immediately" ?
> for example timestamp to a fixed date near 1.1.1970

Wietse proposal is a one-byte record in the queue that can be
updated to "bounce-me".  The bounce reason is written to a separate
file stored in a suitable directory that shares the queue-file's
name.

This is easier than generating the per-recipient bounce logs, ...
It requires reserving 2-bytes in each queue file: a "don't yet
bounce me" record type and zero record length.  The bounce tool
would update the record type to "bounce me" (a 1 byte marker
signalling this).

Best practice might be to put candidate files on hold first, and
bounce them from the hold, rather than active or deferred queues.
This reduces somewhat the opportunities for race conditions when
short (traditional) queue ids are used.

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Bounce mails manually

@lbutlr
In reply to this post by Noel Jones-2

On 26 Feb 2014, at 07:24 , Noel Jones <[hidden email]> wrote:

> On 2/26/2014 12:41 AM, Andreas Schulze wrote:
>>
>> wietse:
>>
>>> I don't know what people are asking for:
>>>  1 - Bounce all recipients of one specific queue file
>>>  2 - Bouncing only specific recipients
>>
>> option 1 (for me)
>>
>> in case of trouble I do
>> - mailq for visual overview
>> - pfqgrep -r/-s address -i | postsuper -d -

OK, what is pfqgrep? I don't see it in my ports tree?

>> In this context it would sometimes be useful to
>> postbounce <queue-id> <reason>
>>
>> Andreas
>
> Same here, option 1. It would occasionally be useful to bounce all
> undelivered recipients of a queue file. But this isn't something I'd
> use every week.

Well, not every week. But there are weeks where I would use it 1000 times.

OK, probably not since postscreen. Still, I had to postsuper -d a couple of dozen messages just a few days ago, and I'm about as low-volume as a server gets.

--
When someone is impatient and says, "I haven't got all day," I always
wonder, How can that be? How can you not have all day?

12