double-bounce check applied to itself

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

double-bounce check applied to itself

Eugene Podshivalov
When reject_unverified_sender param is set and an email is sent on behalf of the server the double-bounce check is still performed (i.e. sent to itself).

Is this all right?

Eugene


Reply | Threaded
Open this post in threaded view
|

Re: double-bounce check applied to itself

Wietse Venema
Eugene Podshivalov:
> When reject_unverified_sender param is set and an email is sent on behalf
> of the server the double-bounce check is still performed (i.e. sent to
> itself).

What is 'the double-bounce check'?

Postfix probes use a sender address that does not receive email.
There is even a feature to make the probe sender time-dependent,
to limit the impact of spam.

        Wietse

address_verify_sender_ttl (default: 0s)
       The time between changes in the time-dependent portion of address veri-
       fication probe sender addresses. The time-dependent portion is appended
       to the  localpart  of  the  address  specified  with  the  address_ver-
       ify_sender  parameter.  This  feature  is ignored when the probe sender
       addresses is the null sender, i.e. the address_verify_sender  value  is
       empty or <>.

       Historically,  the probe sender address was fixed. This has caused such
       addresses to end up on spammer  mailing  lists,  and  has  resulted  in
       wasted network and processing resources.

       To  enable  time-dependent  probe  sender addresses, specify a non-zero
       time value (an integral value plus an optional one-letter  suffix  that
       specifies  the  time unit).  Specify a value of at least several hours,
       to avoid problems with senders that use greylisting.   Avoid  nice  TTL
       values,  to  make the result less predictable.  Time units are: s (sec-
       onds), m (minutes), h (hours), d (days), w (weeks).

       This feature is available in Postfix 2.9 and later.

Reply | Threaded
Open this post in threaded view
|

Re: double-bounce check applied to itself

Eugene Podshivalov
I meant Postfix probes use a sender address even when it is a local one.
Example from logs:
postfix/qmgr[20192]: 9AE7A3F56E: from=<[hidden email]>, size=269, nrcpt=1 (queue active)
postfix/local[20230]: 9AE7A3F56E: to=<[hidden email]>, relay=local, delay=0.02, delays=0.01/0.01/0/0, dsn=2.0.0, status=deliverable (delivers to mailbox)

чт, 11 февр. 2021 г. в 20:58, Wietse Venema <[hidden email]>:
Eugene Podshivalov:
> When reject_unverified_sender param is set and an email is sent on behalf
> of the server the double-bounce check is still performed (i.e. sent to
> itself).

What is 'the double-bounce check'?

Postfix probes use a sender address that does not receive email.
There is even a feature to make the probe sender time-dependent,
to limit the impact of spam.

        Wietse

address_verify_sender_ttl (default: 0s)
       The time between changes in the time-dependent portion of address veri-
       fication probe sender addresses. The time-dependent portion is appended
       to the  localpart  of  the  address  specified  with  the  address_ver-
       ify_sender  parameter.  This  feature  is ignored when the probe sender
       addresses is the null sender, i.e. the address_verify_sender  value  is
       empty or <>.

       Historically,  the probe sender address was fixed. This has caused such
       addresses to end up on spammer  mailing  lists,  and  has  resulted  in
       wasted network and processing resources.

       To  enable  time-dependent  probe  sender addresses, specify a non-zero
       time value (an integral value plus an optional one-letter  suffix  that
       specifies  the  time unit).  Specify a value of at least several hours,
       to avoid problems with senders that use greylisting.   Avoid  nice  TTL
       values,  to  make the result less predictable.  Time units are: s (sec-
       onds), m (minutes), h (hours), d (days), w (weeks).

       This feature is available in Postfix 2.9 and later.

Reply | Threaded
Open this post in threaded view
|

Re: double-bounce check applied to itself

Wietse Venema
Eugene Podshivalov:
> I meant Postfix probes use a sender address even when it is a local one.
> Example from logs:
>
> > postfix/qmgr[20192]: 9AE7A3F56E: from=<[hidden email]>,
> > size=269, nrcpt=1 (queue active)
> > postfix/local[20230]: 9AE7A3F56E: to=<[hidden email]>, *relay=local*,
> > delay=0.02, delays=0.01/0.01/0/0, dsn=2.0.0, status=deliverable (delivers
> > to mailbox)

Do you think that probes should use a DIFFERENT sender address for
some domains? What problem would that solve?

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: double-bounce check applied to itself

Eugene Podshivalov
Let me put it this way: does Postfix do probe for outgoing mail?

чт, 11 февр. 2021 г. в 21:35, Wietse Venema <[hidden email]>:
Eugene Podshivalov:
> I meant Postfix probes use a sender address even when it is a local one.
> Example from logs:
>
> > postfix/qmgr[20192]: 9AE7A3F56E: from=<[hidden email]>,
> > size=269, nrcpt=1 (queue active)
> > postfix/local[20230]: 9AE7A3F56E: to=<[hidden email]>, *relay=local*,
> > delay=0.02, delays=0.01/0.01/0/0, dsn=2.0.0, status=deliverable (delivers
> > to mailbox)

Do you think that probes should use a DIFFERENT sender address for
some domains? What problem would that solve?

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: double-bounce check applied to itself

Wietse Venema
Eugene Podshivalov:
> Let me put it this way: does Postfix do probe for outgoing mail?

reject_unverified_recipient and reject_unverified_sender make no
such distinction. That is a feature, not a bug.

reject_unverified_recipient has been used on internet gateways that
have no complete table of all valid internal recipients. Those
gateways rely on reject_unverified_recipient to probe an internal
system (such as Exchange) and to populate the valid address cache
dynamically.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: double-bounce check applied to itself

Eugene Podshivalov
Assume reject_unverified_sender is set and an email is sent [hidden email].
When the email is sent directly from mail.mydomain.com there is no probe, right?
But when the message is sent from another server that uses mydomain.com as relay then the probe is done, in which case Postfix probes itself.

чт, 11 февр. 2021 г. в 22:58, Wietse Venema <[hidden email]>:
Eugene Podshivalov:
> Let me put it this way: does Postfix do probe for outgoing mail?

reject_unverified_recipient and reject_unverified_sender make no
such distinction. That is a feature, not a bug.

reject_unverified_recipient has been used on internet gateways that
have no complete table of all valid internal recipients. Those
gateways rely on reject_unverified_recipient to probe an internal
system (such as Exchange) and to populate the valid address cache
dynamically.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: double-bounce check applied to itself

Viktor Dukhovni
> On Feb 11, 2021, at 6:29 PM, Eugene Podshivalov <[hidden email]> wrote:
>
> Assume reject_unverified_sender is set and an email is sent From:[hidden email].

This is an smtpd(8)/access(5) feature, and so only applies when email is
received via SMTP and the restriction in question is applied to the message,
(i.e. is not short-circuited by some earlier "OK" action).  It cannot apply
when email is submitted locally via sendmail(1)/postdrop(1).

> When the email is sent directly from mail.mydomain.com there is no probe, right?

See above.

> But when the message is sent from another server that uses mydomain.com as relay
> then the probe is done, in which case Postfix probes itself.

Probes verify whether an input address is plausibly valid by simulating
delivery.  The recipient is handed off to some transport, which reports
success ("deliverable") or failure.  There's no distinction here between
local and remote transports.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: double-bounce check applied to itself

Wietse Venema
In reply to this post by Eugene Podshivalov
Eugene Podshivalov:
> Assume reject_unverified_sender is set and an email is sent
> From:[hidden email].
> When the email is sent directly from mail.mydomain.com there is no probe,
> right?

reject_unverified_recipient etc. do not care where mail comes from,
or where it is being sent to.

> But when the message is sent from another server that uses mydomain.com as
> relay then the probe is done, in which case Postfix probes itself.

And it should. If an email address DOMAIN is local, that does not
mean that the MAILBOX will be local. The address can be transformed
with canonical_maps, virtual_alias_maps, it may be routed to a
different system with transport_maps, and it may be aliased with
/etc/aliases to some other local or remote address. Those things
can be 'verified' only by sending a probe message through Postfix.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: double-bounce check applied to itself

Eugene Podshivalov
Wietse:
The address can be transformed
with canonical_maps, virtual_alias_maps, it may be routed to a
different system with transport_maps, and it may be aliased with
/etc/aliases to some other local or remote address
All these things apply to locally sent messages as well, don't they? But still no probe is done for them.
Is it because local messages are considered more trusted then relay ones, or maybe some other silent verification mechanisms are utilized for them which cannot be used for relay ones?

чт, 11 февр. 2021 г. в 23:48, Wietse Venema <[hidden email]>:
Eugene Podshivalov:
> Assume reject_unverified_sender is set and an email is sent
> [hidden email].
> When the email is sent directly from mail.mydomain.com there is no probe,
> right?

reject_unverified_recipient etc. do not care where mail comes from,
or where it is being sent to.

> But when the message is sent from another server that uses mydomain.com as
> relay then the probe is done, in which case Postfix probes itself.

And it should. If an email address DOMAIN is local, that does not
mean that the MAILBOX will be local. The address can be transformed
with canonical_maps, virtual_alias_maps, it may be routed to a
different system with transport_maps, and it may be aliased with
/etc/aliases to some other local or remote address. Those things
can be 'verified' only by sending a probe message through Postfix.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: double-bounce check applied to itself

Wietse Venema
Eugene Podshivalov:

> >
> > Wietse:
>
> The address can be transformed
> > with canonical_maps, virtual_alias_maps, it may be routed to a
> > different system with transport_maps, and it may be aliased with
> > /etc/aliases to some other local or remote address
>
> All these things apply to locally sent messages as well, don't they? But
> still no probe is done for them.

There are no smtpd_mumble_restrictions for /usr/sbin/sendmail,
because rejecting mail there would totally break compatibility with
every program that submits mail that way.

Instead, Postfix accepts mail, and LATER returns undeliverable mail
to the alleged sender. Local users can be punished after an analysis
of the logs of their activty.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: double-bounce check applied to itself

Eugene Podshivalov
Have read through the pickup(8) and cleanup(8) docs and understood the workflow as follows.
When a local message is sent by sendmail it lands into the maildrop directory. The pickup daemon then feeds it into the cleanup one.
The latter is responsible for all sorts of envelope and header transformations including virtual and canonical mappings resolution.

pickup[18650]: D30B93F909: uid=1000 from=<[hidden email]>
cleanup[19082]: D30B93F909: message-id=<[hidden email]>
qmgr[16657]: D30B93F909: from=<[hidden email]>, size=379, nrcpt=1 (queue active)
smtp[19084]: D30B93F909: to=<[hidden email]>, relay=gmail-smtp-in.l.google.com[64.233.162.27]:25, delay=0.87, delays=0.03/0.01/0.35/0.49, dsn=2.0.0, status=sent (250 2.0.0 OK  1613056017 z11si3995053lfe.163 - gsmtp)
qmgr[16657]: D30B93F909: removed

 Relay messages are also feeded into the cleanup daemon but by smtpd which initiates the probe prior to that.

smtpd[20221]: connect from mail-yb1-f170.google.com[209.85.219.170]
smtpd[20221]: Anonymous TLS connection established from mail-yb1-f170.google.com[209.85.219.170]: TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
cleanup[20229]: 9AE7A3F56E: message-id=<[hidden email]>
qmgr[20192]: 9AE7A3F56E: from=<[hidden email]>, size=269, nrcpt=1 (queue active)
local[20230]: 9AE7A3F56E: to=<[hidden email]>, relay=local, delay=0.02, delays=0.01/0.01/0/0, dsn=2.0.0, status=deliverable (delivers to mailbox)
qmgr[20192]: 9AE7A3F56E: removed
smtpd[20221]: 9A4A73F56E: client=mail-yb1-f170.google.com[209.85.219.170], sasl_method=PLAIN, sasl_username=user
cleanup[20229]: 9A4A73F56E: message-id=<[hidden email]>
qmgr[20192]: 9A4A73F56E: from=<[hidden email]>, size=1489, nrcpt=1 (queue active)
smtpd[20221]: disconnect from mail-yb1-f170.google.com[209.85.219.170] ehlo=2 starttls=1 auth=1 mail=1 rcpt=1 data=1 quit=1 commands=8
smtp[20231]: 9A4A73F56E: to=<[hidden email]>, relay=gmail-smtp-in.l.google.com[64.233.162.27]:25, delay=0.89, delays=0.29/0.01/0.32/0.27, dsn=2.0.0, status=sent (250 2.0.0 OK  1613059176 c9si1597016lft.196 - gsmtp)
qmgr[20192]: 9A4A73F56E: removed

So now it is clear why the probe is not done for local mail.

Another somewhat related question is: in order to probe the smtpd needs to resolve all virtual etc. mappings which is also done by the cleanup. Is this resolution done twice in this case?



пт, 12 февр. 2021 г. в 02:48, Wietse Venema <[hidden email]>:
Eugene Podshivalov:
> >
> > Wietse:
>
> The address can be transformed
> > with canonical_maps, virtual_alias_maps, it may be routed to a
> > different system with transport_maps, and it may be aliased with
> > /etc/aliases to some other local or remote address
>
> All these things apply to locally sent messages as well, don't they? But
> still no probe is done for them.

There are no smtpd_mumble_restrictions for /usr/sbin/sendmail,
because rejecting mail there would totally break compatibility with
every program that submits mail that way.

Instead, Postfix accepts mail, and LATER returns undeliverable mail
to the alleged sender. Local users can be punished after an analysis
of the logs of their activty.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: double-bounce check applied to itself

Wietse Venema
Eugene Podshivalov:
> Another somewhat related question is: in order to probe the smtpd needs to
> resolve all virtual etc. mappings which is also done by the cleanup. Is
> this resolution done twice in this case?

smtpd does not resolve any mappings.

Instead it makes a safe guess to reject most non-existent recipients.
This guess must be safe because it must not reject any valid recipient.

        Wietse

Reply | Threaded
Open this post in threaded view
|

Re: double-bounce check applied to itself

Eugene Podshivalov
Wietse:
smtpd does not resolve any mappings.
If a sender address is mapped to some other virtual alias, how does it know which address to probe then?

Без вирусов. www.avast.ru

пт, 12 февр. 2021 г. в 19:32, Wietse Venema <[hidden email]>:
Eugene Podshivalov:
> Another somewhat related question is: in order to probe the smtpd needs to
> resolve all virtual etc. mappings which is also done by the cleanup. Is
> this resolution done twice in this case?

smtpd does not resolve any mappings.

Instead it makes a safe guess to reject most non-existent recipients.
This guess must be safe because it must not reject any valid recipient.

        Wietse

Reply | Threaded
Open this post in threaded view
|

Re: double-bounce check applied to itself

Eugene Podshivalov
Judging by the above log example the double-bounce message is processed by the cleanup as well. I guess that's the point where the resolving happens for the prob.

Без вирусов. www.avast.ru

пт, 12 февр. 2021 г. в 19:43, Eugene Podshivalov <[hidden email]>:
Wietse:
smtpd does not resolve any mappings.
If a sender address is mapped to some other virtual alias, how does it know which address to probe then?

Без вирусов. www.avast.ru

пт, 12 февр. 2021 г. в 19:32, Wietse Venema <[hidden email]>:
Eugene Podshivalov:
> Another somewhat related question is: in order to probe the smtpd needs to
> resolve all virtual etc. mappings which is also done by the cleanup. Is
> this resolution done twice in this case?

smtpd does not resolve any mappings.

Instead it makes a safe guess to reject most non-existent recipients.
This guess must be safe because it must not reject any valid recipient.

        Wietse

Reply | Threaded
Open this post in threaded view
|

Re: double-bounce check applied to itself

Wietse Venema
In reply to this post by Eugene Podshivalov
Eugene Podshivalov:
> >
> > Wietse:
> > smtpd does not resolve any mappings.
>
> If a sender address is mapped to some other virtual alias, how does it know
> which address to probe then?

The Postfix SMTP daemon generates a probe message for the address
in the RCPT TO command. Inside Postfix, that probe message will
have the same address mappings etc. as a real message for that
recipient, except that the probe message is not delivered.

We are going around in a circle two times now. Goodbye.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: double-bounce check applied to itself

Viktor Dukhovni
In reply to this post by Eugene Podshivalov
> On Feb 12, 2021, at 12:29 PM, Eugene Podshivalov <[hidden email]> wrote:
>
> Another somewhat related question is: in order to probe the smtpd needs to resolve all virtual etc. mappings which is also done by the cleanup. Is this resolution done twice in this case?

The smtpd(8) process does not perform any *rewriting*, the unmodified
"MAIL FROM" and "RCPT TO" addresses are passed to cleanup(8).

However, smtpd(8) may perform a virtual(8) *lookup* in order to determine
recipient address *existence*, as part of table-based recipient address
validation (not to be confused with probe-based recipient address
verification).

  - virtual_alias_maps for all non-default address classes
  - local_recipient_maps for addresses matching $mydestination
  - virtual_mailbox_maps for addresses matching $virtual_mailbox_domains
  - relay_recipient_maps for addresses matching $relay_domains

These lookups check *existence* only.  The RHS value is discarded.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: double-bounce check applied to itself

Wietse Venema
Viktor Dukhovni:

> > On Feb 12, 2021, at 12:29 PM, Eugene Podshivalov <[hidden email]> wrote:
> >
> > Another somewhat related question is: in order to probe the smtpd needs to resolve all virtual etc. mappings which is also done by the cleanup. Is this resolution done twice in this case?
>
> The smtpd(8) process does not perform any *rewriting*, the unmodified
> "MAIL FROM" and "RCPT TO" addresses are passed to cleanup(8).
>
> However, smtpd(8) may perform a virtual(8) *lookup* in order to determine
> recipient address *existence*, as part of table-based recipient address
> validation (not to be confused with probe-based recipient address
> verification).
>
>   - virtual_alias_maps for all non-default address classes
>   - local_recipient_maps for addresses matching $mydestination
>   - virtual_mailbox_maps for addresses matching $virtual_mailbox_domains
>   - relay_recipient_maps for addresses matching $relay_domains
>
> These lookups check *existence* only.  The RHS value is discarded.

Also: {recipient,}canonical_maps for all address classes.

        Wietse