bounced posts go to spam

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

bounced posts go to spam

Sonic
Hello,

I have a simple relay for sending emails from internal scanners and a
voicemail system. All works fine except for posts that get bounced as
the bounce notifications somehow fail both SPF and DKIM tests.

The only (seemingly significant) differences I can find in the headers
of normal vs bounced posts from the system are:

Normal:
Authentication-Results: test17.example.com; spf=pass
smtp.mailfrom=[hidden email]
Authentication-Results: test17.example.com; dkim=pass (2048-bit key)

Bounced:
Authentication-Results: test17.example.com; spf=none smtp.helo=smtp.example.com
Authentication-Results: test17.example.com; dkim=none

The normal mail has:
smtp.mailfrom=[hidden email]
and the bounced mail has:
smtp.helo=smtp.example.com

And so it looks like this difference is keeping the bounced
notifications from passing SPF and getting processed by OpenDKIM.

Suggestions?

Thanks!
Chris
Reply | Threaded
Open this post in threaded view
|

Re: bounced posts go to spam

Matus UHLAR - fantomas
On 30.07.18 15:22, Sonic wrote:
>I have a simple relay for sending emails from internal scanners and a
>voicemail system. All works fine except for posts that get bounced as
>the bounce notifications somehow fail both SPF and DKIM tests.

please provide more info about the mail flow.

does your simple relay reject the mail, does your server reject the mail
when receiving from the relay, or do remote servers reject the mail from
your simple relay?

Note that "bounce" happens when mail server receives a mail, but is unable
to deliver it, so it constructs a bounce and sends is "back".

the bounce itself should not trigger SPF (since the envelope from is empty) nor DKIM
(unless server creating the bounce uses a domain that it can't sign)

>The only (seemingly significant) differences I can find in the headers
>of normal vs bounced posts from the system are:
>
>Normal:
>Authentication-Results: test17.example.com; spf=pass
>smtp.mailfrom=[hidden email]
>Authentication-Results: test17.example.com; dkim=pass (2048-bit key)
>
>Bounced:
>Authentication-Results: test17.example.com; spf=none smtp.helo=smtp.example.com
>Authentication-Results: test17.example.com; dkim=none

"none" means no result. It does not mean spf or dkim failed.
they did not fail, neither one.

>The normal mail has:
>smtp.mailfrom=[hidden email]
>and the bounced mail has:
>smtp.helo=smtp.example.com

mailfrom is different than helo. you are comparing apples and oranges.

>And so it looks like this difference is keeping the bounced
>notifications from passing SPF and getting processed by OpenDKIM.

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Enter any 12-digit prime number to continue.
Reply | Threaded
Open this post in threaded view
|

Re: bounced posts go to spam

Sonic
> does your simple relay reject the mail, does your server reject the mail
> when receiving from the relay, or do remote servers reject the mail from
> your simple relay?

The remote servers reject, or place in spam, bounced and NDR's from
the relay, due to a strict DMARC policy.

> Note that "bounce" happens when mail server receives a mail, but is unable
> to deliver it, so it constructs a bounce and sends is "back".
>
> the bounce itself should not trigger SPF (since the envelope from is empty)
> nor DKIM
> (unless server creating the bounce uses a domain that it can't sign)

Apparently internally generated email by Postfix does not go through
the milter and therefore does not get signed by OpenDKIM.
It also appears to come from a sub-domain, the HELO name, and not just
the SLD (in this particular case) which causes it to fail SPF as well
(and possibly because of this wouldn't get signed by the milter if it
was directed through it).
Reply | Threaded
Open this post in threaded view
|

Re: bounced posts go to spam

Wietse Venema
Sonic:

> > does your simple relay reject the mail, does your server reject the mail
> > when receiving from the relay, or do remote servers reject the mail from
> > your simple relay?
>
> The remote servers reject, or place in spam, bounced and NDR's from
> the relay, due to a strict DMARC policy.
>
> > Note that "bounce" happens when mail server receives a mail, but is unable
> > to deliver it, so it constructs a bounce and sends is "back".
> >
> > the bounce itself should not trigger SPF (since the envelope from is empty)
> > nor DKIM
> > (unless server creating the bounce uses a domain that it can't sign)
>
> Apparently internally generated email by Postfix does not go through
> the milter and therefore does not get signed by OpenDKIM.

Try setting

/etc/postfix/main.cf:
    internal_mail_filter_classes = bounce

(this assumes that you have configured "non_smtpd_milters" to invoke
the DKIM signer).

> It also appears to come from a sub-domain, the HELO name, and not just
> the SLD (in this particular case) which causes it to fail SPF as well

The sender domain is condigured with myorigin, you need to change
that if you want the domain instead.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: bounced posts go to spam

Sonic
> Try setting
>
> /etc/postfix/main.cf:
>     internal_mail_filter_classes = bounce
>
> (this assumes that you have configured "non_smtpd_milters" to invoke
> the DKIM signer).
>
>> It also appears to come from a sub-domain, the HELO name, and not just
>> the SLD (in this particular case) which causes it to fail SPF as well
>
> The sender domain is condigured with myorigin, you need to change
> that if you want the domain instead.

Hi Wietse,

That works in one case but not another.

If I attempt to send from a domain whose DMARC policies do not allow
sending from this server, the sender will now receive the NDR in the
inbox as it (the NDR) meets the SPF/DKIM tests:
==========================================================================
Jul 31 10:43:26 eserver postfix/pickup[20439]: F02ED403E25: uid=0
from=<[hidden email]>
Jul 31 10:43:26 eserver postfix/cleanup[20674]: F02ED403E25:
message-id=<[hidden email]>
Jul 31 10:43:27 eserver postfix/qmgr[20440]: F02ED403E25:
from=<[hidden email]>, size=465, nrcpt=1 (queue active)
Jul 31 10:43:27 eserver postfix/smtp[20676]: F02ED403E25:
to=<[hidden email]>, relay=ASPMX.L.GOOGLE.com[74.125.202.27]:25,
delay=0.59, delays=0.07/0.01/0.25/0.27, dsn=5.7.1, status=bounced (host
ASPMX.L.GOOGLE.com[74.125.202.27] said: 550-5.7.1 Unauthenticated email from
example.net is not accepted due to 550-5.7.1 domain's DMARC policy. Please
contact the administrator of 550-5.7.1 example.net domain if this was a
legitimate mail. Please visit 550-5.7.1
https://support.google.com/mail/answer/2451690 to learn about the 550 5.7.1
DMARC initiative. x23-v6si1859094ita.142 - gsmtp (in reply to end of DATA
command))
Jul 31 10:43:27 eserver postfix/cleanup[20674]: 8A897403E24:
message-id=<[hidden email]>
Jul 31 10:43:27 eserver postfix/bounce[20677]: F02ED403E25: sender non-delivery
notification: 8A897403E24
Jul 31 10:43:27 eserver postfix/qmgr[20440]: 8A897403E24: from=<>, size=3329,
nrcpt=1 (queue active)
Jul 31 10:43:27 eserver postfix/qmgr[20440]: F02ED403E25: removed
Jul 31 10:43:30 eserver postfix/smtp[20676]: 8A897403E24:
to=<[hidden email]>, relay=mail.example.org[185.70.40.101]:25,
delay=2.6, delays=0.03/0/1/1.5, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as
AD72C92)
Jul 31 10:43:30 eserver postfix/qmgr[20440]: 8A897403E24: removed
============================================================================

However if I send from a valid user account to an address that causes
a bounce (non-existent in this case), the NDR gets rejected by the
senders email service:
============================================================================
Jul 31 10:17:45 eserver postfix/pickup[19900]: 511AE403E25: uid=0
from=<[hidden email]>
Jul 31 10:17:45 eserver postfix/cleanup[19977]: 511AE403E25:
message-id=<[hidden email]>
Jul 31 10:17:45 eserver postfix/qmgr[19901]: 511AE403E25:
from=<[hidden email]>, size=523, nrcpt=1 (queue active)
Jul 31 10:17:46 eserver postfix/smtp[19978]: 511AE403E25:
to=<[hidden email]>, relay=mail.example.org[185.70.40.101]:25,
delay=1.3, dela
ys=0.05/0/1/0.17, dsn=5.7.1, status=bounced (host
mail.example.org[185.70.40.101] said: 554 5.7.1 <[hidden email]>:
Recipient address rej
ected: this address does not exist (in reply to RCPT TO command))

Jul 31 10:17:46 eserver postfix/cleanup[19977]: 88382403E24:
message-id=<[hidden email]>
  Jul 31 10:17:46 eserver postfix/bounce[19981]: 511AE403E25: sender
non-delivery notification: 88382403E24                                       Jul
31 10:17:46 eserver postfix/qmgr[19901]: 88382403E24: from=<>, size=3359,
nrcpt=1 (queue active)                                            Jul 31
10:17:46 eserver postfix/qmgr[19901]: 511AE403E25: removed
                                                         Jul 31 10:17:47
eserver postfix/smtp[19978]: 88382403E24: to=<[hidden email]>,
relay=ASPMX.L.GOOGLE.com[74.125.202.27]:25, delay=0.47,
 delays=0.02/0/0.23/0.22, dsn=5.7.1, status=bounced (host
ASPMX.L.GOOGLE.com[74.125.202.27] said: 550-5.7.1 Unauthenticated email from
example
.com is not accepted due to 550-5.7.1 domain's DMARC policy. Please contact the
administrator of 550-5.7.1 example.com domain if this was a le
gitimate mail. Please visit 550-5.7.1
https://support.google.com/mail/answer/2451690 to learn about the 550 5.7.1
DMARC initiative. o20-v6si1040
1393iod.272 - gsmtp (in reply to end of DATA command))

Jul 31 10:17:47 eserver postfix/qmgr[19901]: 88382403E24: removed
============================================================================

Of course the names have been changed to protect the guilty :-)

I don't see why the NDR in the second case should fail DMARC, when it
passes in the first case.

Chris
Reply | Threaded
Open this post in threaded view
|

Re: bounced posts go to spam

Wietse Venema
Sonic:

> > Try setting
> >
> > /etc/postfix/main.cf:
> >     internal_mail_filter_classes = bounce
> >
> > (this assumes that you have configured "non_smtpd_milters" to invoke
> > the DKIM signer).
> >
> >> It also appears to come from a sub-domain, the HELO name, and not just
> >> the SLD (in this particular case) which causes it to fail SPF as well
> >
> > The sender domain is condigured with myorigin, you need to change
> > that if you want the domain instead.
>
> Hi Wietse,
>
> That works in one case but not another.
>
> If I attempt to send from a domain whose DMARC policies do not allow
> sending from this server, the sender will now receive the NDR in the
> inbox as it (the NDR) meets the SPF/DKIM tests:
> ==========================================================================
> Jul 31 10:43:26 eserver postfix/pickup[20439]: F02ED403E25: uid=0
> from=<[hidden email]>
> Jul 31 10:43:26 eserver postfix/cleanup[20674]: F02ED403E25:
> message-id=<[hidden email]>
> Jul 31 10:43:27 eserver postfix/qmgr[20440]: F02ED403E25:
> from=<[hidden email]>, size=465, nrcpt=1 (queue active)
> Jul 31 10:43:27 eserver postfix/smtp[20676]: F02ED403E25:
> to=<[hidden email]>, relay=ASPMX.L.GOOGLE.com[74.125.202.27]:25,
> delay=0.59, delays=0.07/0.01/0.25/0.27, dsn=5.7.1, status=bounced (host
> ASPMX.L.GOOGLE.com[74.125.202.27] said: 550-5.7.1 Unauthenticated email from
> example.net is not accepted due to 550-5.7.1 domain's DMARC policy. Please
> contact the administrator of 550-5.7.1 example.net domain if this was a
> legitimate mail. Please visit 550-5.7.1
> https://support.google.com/mail/answer/2451690 to learn about the 550 5.7.1
> DMARC initiative. x23-v6si1859094ita.142 - gsmtp (in reply to end of DATA
> command))
> Jul 31 10:43:27 eserver postfix/cleanup[20674]: 8A897403E24:
> message-id=<[hidden email]>
> Jul 31 10:43:27 eserver postfix/bounce[20677]: F02ED403E25: sender non-delivery
> notification: 8A897403E24
> Jul 31 10:43:27 eserver postfix/qmgr[20440]: 8A897403E24: from=<>, size=3329,
> nrcpt=1 (queue active)
> Jul 31 10:43:27 eserver postfix/qmgr[20440]: F02ED403E25: removed
> Jul 31 10:43:30 eserver postfix/smtp[20676]: 8A897403E24:
> to=<[hidden email]>, relay=mail.example.org[185.70.40.101]:25,
> delay=2.6, delays=0.03/0/1/1.5, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as
> AD72C92)
> Jul 31 10:43:30 eserver postfix/qmgr[20440]: 8A897403E24: removed

This bounce message is sent to mail.example.org.

> ============================================================================
>
> However if I send from a valid user account to an address that causes
> a bounce (non-existent in this case), the NDR gets rejected by the
> senders email service:
> ============================================================================
> Jul 31 10:17:45 eserver postfix/pickup[19900]: 511AE403E25: uid=0
> from=<[hidden email]>
> Jul 31 10:17:45 eserver postfix/cleanup[19977]: 511AE403E25:
> message-id=<[hidden email]>
> Jul 31 10:17:45 eserver postfix/qmgr[19901]: 511AE403E25:
> from=<[hidden email]>, size=523, nrcpt=1 (queue active)
> Jul 31 10:17:46 eserver postfix/smtp[19978]: 511AE403E25:
> to=<[hidden email]>, relay=mail.example.org[185.70.40.101]:25,
> delay=1.3, dela
> ys=0.05/0/1/0.17, dsn=5.7.1, status=bounced (host
> mail.example.org[185.70.40.101] said: 554 5.7.1 <[hidden email]>:
> Recipient address rej
> ected: this address does not exist (in reply to RCPT TO command))
>
> Jul 31 10:17:46 eserver postfix/cleanup[19977]: 88382403E24:
> message-id=<[hidden email]>
>   Jul 31 10:17:46 eserver postfix/bounce[19981]: 511AE403E25: sender
> non-delivery notification: 88382403E24                                       Jul
> 31 10:17:46 eserver postfix/qmgr[19901]: 88382403E24: from=<>, size=3359,
> nrcpt=1 (queue active)                                            Jul 31
> 10:17:46 eserver postfix/qmgr[19901]: 511AE403E25: removed
>                                                          Jul 31 10:17:47
> eserver postfix/smtp[19978]: 88382403E24: to=<[hidden email]>,
> relay=ASPMX.L.GOOGLE.com[74.125.202.27]:25, delay=0.47,
>  delays=0.02/0/0.23/0.22, dsn=5.7.1, status=bounced (host
> ASPMX.L.GOOGLE.com[74.125.202.27] said: 550-5.7.1 Unauthenticated email from
> example
> .com is not accepted due to 550-5.7.1 domain's DMARC policy. Please contact the
> administrator of 550-5.7.1 example.com domain if this was a le
> gitimate mail. Please visit 550-5.7.1
> https://support.google.com/mail/answer/2451690 to learn about the 550 5.7.1
> DMARC initiative. o20-v6si1040
> 1393iod.272 - gsmtp (in reply to end of DATA command))
>
> Jul 31 10:17:47 eserver postfix/qmgr[19901]: 88382403E24: removed
> ============================================================================

This bounce message was sent to ASPMX.L.GOOGLE.com.

Apparently, mail.example.org and ASPMX.L.GOOGLE.com enforce DMARC
in different ways.

Regardless, if the DMARC policy does not authorize host Y to send
mail on behalf of domain $myorigin, then you need to fix the DMARC
policy so that those bounces sent by host Y aren't violating DMARC,
or you need to somehow route those bounces from host Y through a
host that is DMARC-authorized.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: bounced posts go to spam

Sonic
> Apparently, mail.example.org and ASPMX.L.GOOGLE.com enforce DMARC
> in different ways.
>
> Regardless, if the DMARC policy does not authorize host Y to send
> mail on behalf of domain $myorigin, then you need to fix the DMARC
> policy so that those bounces sent by host Y aren't violating DMARC,
> or you need to somehow route those bounces from host Y through a
> host that is DMARC-authorized.

All normal mail gets delivered just fine. The domain in question
(example.com) has an SPF record including the server's (outside) IP
address (and proper A and PTR records), and OpenDKIM signs all regular
email.
Examining the headers of all normal (non-NDR) post receipts show they
pass both SPF, and DKIM tests and therefore DMARC as well. Plus the
majority of sent posts are to the Google servers (with no issues).
It's only the bounces/NDR's that have an issue.

Thanks,

Chris
Reply | Threaded
Open this post in threaded view
|

Re: bounced posts go to spam

Dominic Raferd


On Tue, 31 Jul 2018 at 16:52, Sonic <[hidden email]> wrote:
> Apparently, mail.example.org and ASPMX.L.GOOGLE.com enforce DMARC
> in different ways.
>
> Regardless, if the DMARC policy does not authorize host Y to send
> mail on behalf of domain $myorigin, then you need to fix the DMARC
> policy so that those bounces sent by host Y aren't violating DMARC,
> or you need to somehow route those bounces from host Y through a
> host that is DMARC-authorized.

All normal mail gets delivered just fine. The domain in question
(example.com) has an SPF record including the server's (outside) IP
address (and proper A and PTR records), and OpenDKIM signs all regular
email.
Examining the headers of all normal (non-NDR) post receipts show they
pass both SPF, and DKIM tests and therefore DMARC as well. Plus the
majority of sent posts are to the Google servers (with no issues).
It's only the bounces/NDR's that have an issue.

Maybe this piece of magic (suggested by Wietse a while ago) might help - it's a way to overcome double_bounce_sender having @$myhostname auto-added:

canonical_maps = inline:{$double_bounce_sender@$myhostname=double-bounce@$mydomain}  [...]