Quantcast

problem with protection.outlook.com released spam getting bounced

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

problem with protection.outlook.com released spam getting bounced

John Stoffel-2

Hi all,

We're running postfix-2.6.6-6.el6_5.x86_64 on RHEL 6.6 and running
into a problem where emails that have been released from our outside
spam protection company, *.protection.outlook.com, are getting
rejected with messages like this:

  Mar 26 06:00:56 mailhost postfix/smtpd[2270]: connect from mail-sn1nam01lp0113.outbound.protection.outlook.com[207.46.163.113]
  Mar 26 06:00:56 mailhost postfix/smtpd[2270]: 51235A07D1: client=mail-sn1nam01lp0113.outbound.protection.outlook.com[207.46.163.113]
  Mar 26 06:00:56 mailhost postfix/cleanup[2279]: 51235A07D1: message-id=<[hidden email]>
  Mar 26 06:00:56 mailhost postfix/qmgr[27442]: 51235A07D1: from=<[hidden email]>, size=40439, nrcpt=1 (queue active)
  Mar 26 06:00:56 mailhost postfix/local[2278]: 51235A07D1: to=<[hidden email]>, relay=local, delay=0.29, delays=0.28/0/0/0.01, dsn=5.4.6, status=bounced (mail forwarding loop for [hidden email])
  Mar 26 06:00:56 mailhost postfix/bounce[2273]: 51235A07D1: sender non-delivery notification: 97DF2A080B
  Mar 26 06:00:56 mailhost postfix/qmgr[27442]: 51235A07D1: removed

These emails are released by the end user and should be delivered, but are getting bounced back.

How would I go about figuring out if it's really a bogus "Delivered-To: " header that's causing this rejection?  



# postconf -n
alias_database = hash:/etc/aliases
alias_maps = nis:mail.aliases
command_directory = /usr/sbin
config_directory = /etc/postfix
daemon_directory = /usr/libexec/postfix
data_directory = /var/lib/postfix
debug_peer_level = 2
fallback_transport =
html_directory = no
inet_interfaces = all
inet_protocols = ipv4
local_header_rewrite_clients = static:all
local_recipient_maps =
mail_owner = postfix
mailq_path = /usr/bin/mailq.postfix
manpage_directory = /usr/share/man
masquerade_domains = !hqmta.sub.com $myorigin
message_size_limit = 30240000
mydestination = $myhostname, localhost.$mydomain, localhost,
$mydomain, sub.com, acs.sub.corp.com
mydomain = sub.corp.com
myhostname = mailhost.sub.corp.com
mynetworks = 127.0.0.0/8, 209.243.0.0/16, 10.0.0.0/8
myorigin = $mydomain
newaliases_path = /usr/bin/newaliases.postfix
queue_directory = /var/spool/postfix
readme_directory = /usr/share/doc/postfix-2.6.6/README_FILES
relay_domains = $mydestination, other.com, otherfoobar.com
sample_directory = /usr/share/doc/postfix-2.6.6/samples
sender_canonical_maps = hash:/etc/postfix/sender_canonical
sendmail_path = /usr/sbin/sendmail.postfix
setgid_group = postdrop
transport_maps = hash:/etc/postfix/transport_maps
unknown_local_recipient_reject_code = 450
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: problem with protection.outlook.com released spam getting bounced

Dominic Raferd


On 30 March 2017 at 15:26, John Stoffel <[hidden email]> wrote:

Hi all,

We're running postfix-2.6.6-6.el6_5.x86_64 on RHEL 6.6 and running
into a problem where emails that have been released from our outside
spam protection company, *.protection.outlook.com, are getting
rejected with messages like this:

  Mar 26 06:00:56 mailhost postfix/smtpd[2270]: connect from mail-sn1nam01lp0113.outbound.protection.outlook.com[207.46.163.113]
  Mar 26 06:00:56 mailhost postfix/smtpd[2270]: 51235A07D1: client=mail-sn1nam01lp0113.outbound.protection.outlook.com[207.46.163.113]
  Mar 26 06:00:56 mailhost postfix/cleanup[2279]: 51235A07D1: message-id=<[hidden email]>
  Mar 26 06:00:56 mailhost postfix/qmgr[27442]: 51235A07D1: from=<[hidden email]>, size=40439, nrcpt=1 (queue active)
  Mar 26 06:00:56 mailhost postfix/local[2278]: 51235A07D1: to=<[hidden email]>, relay=local, delay=0.29, delays=0.28/0/0/0.01, dsn=5.4.6, status=bounced (mail forwarding loop for [hidden email])
  Mar 26 06:00:56 mailhost postfix/bounce[2273]: 51235A07D1: sender non-delivery notification: 97DF2A080B
  Mar 26 06:00:56 mailhost postfix/qmgr[27442]: 51235A07D1: removed

These emails are released by the end user and should be delivered, but are getting bounced back.

How would I go about figuring out if it's really a bogus "Delivered-To: " header that's causing this rejection?

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: problem with protection.outlook.com released spam getting bounced

John Stoffel-2
>>>>> "Dominic" == Dominic Raferd <[hidden email]> writes:

Dominic> On 30 March 2017 at 15:26, John Stoffel <[hidden email]> wrote:


Dominic>     Hi all,

Dominic>     We're running postfix-2.6.6-6.el6_5.x86_64 on RHEL 6.6 and running
Dominic>     into a problem where emails that have been released from our outside
Dominic>     spam protection company, *.protection.outlook.com, are getting
Dominic>     rejected with messages like this:

Dominic>       Mar 26 06:00:56 mailhost postfix/smtpd[2270]: connect from
Dominic>     mail-sn1nam01lp0113.outbound.protection.outlook.com[207.46.163.113]
Dominic>       Mar 26 06:00:56 mailhost postfix/smtpd[2270]: 51235A07D1: client=
Dominic>     mail-sn1nam01lp0113.outbound.protection.outlook.com[207.46.163.113]
Dominic>       Mar 26 06:00:56 mailhost postfix/cleanup[2279]: 51235A07D1: message-id=<
Dominic>     [hidden email]>
Dominic>       Mar 26 06:00:56 mailhost postfix/qmgr[27442]: 51235A07D1: from=<
Dominic>     [hidden email]>, size=40439, nrcpt=1 (queue active)
Dominic>       Mar 26 06:00:56 mailhost postfix/local[2278]: 51235A07D1: to=<
Dominic>     [hidden email]>, relay=local, delay=0.29, delays=0.28/0/0/0.01, dsn=
Dominic>     5.4.6, status=bounced (mail forwarding loop for [hidden email])
Dominic>       Mar 26 06:00:56 mailhost postfix/bounce[2273]: 51235A07D1: sender
Dominic>     non-delivery notification: 97DF2A080B
Dominic>       Mar 26 06:00:56 mailhost postfix/qmgr[27442]: 51235A07D1: removed

Dominic>     These emails are released by the end user and should be delivered, but are
Dominic>     getting bounced back.

Dominic>     How would I go about figuring out if it's really a bogus "Delivered-To: "
Dominic>     header that's causing this rejection?


Dominic> Did you see this earlier thread: http://postfix.1071664.n5.nabble.com/
Dominic> What-is-causing-this-mail-forwarding-loop-bounce-td62199.html ?

I have looked at that thread, but I don't have a good answer.  I can
understand Wietse's comments in there, but I'm trying to also solve
this type of problem, and I've posted my postconf -n output, and the
logs from my mail host.  

Of course protection.outlook.com says they're not doing anything
special... but I don't believe them.  But I can't prove it without
keeping a copy of the bounced email somehow.  

Is there a good way to log the full headers of these emails before
they get rejected, so I can at least know what's going on here?

Thanks,
John
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: problem with protection.outlook.com released spam getting bounced

Wietse Venema
Postfix reports this error because it is responsible for 'example.com'
and the message has 'Delivered-To: [hidden email]'.

There are several options:

1) Your error. Your system is sending mail out after adding the
'Delivered-To: [hidden email]' header, and it comes back from
outlook.com. If Postfix did not block such email then it could loop
forever.

2) User error. After your system adds the 'Delivered-To: [hidden email]'
header, some of your users forward their email off-site.  That email
ends up at a system that looks at the message header address and
that forwards that email to outlook.com. This results in a similar
problem as (1). If Postfix did not block such email then it could
loop forever.

2) Outlook error. Outlook.com adds the 'Delivered-To: [hidden email]'
header. In that case all mail from Outlook.com would have this
problem. It does not seem likely.

My money is on (1) or (2).

        Wietse

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: problem with protection.outlook.com released spam getting bounced

Noel Jones-2
In reply to this post by John Stoffel-2
On 3/30/2017 9:26 AM, John Stoffel wrote:

>
> Hi all,
>
> We're running postfix-2.6.6-6.el6_5.x86_64 on RHEL 6.6 and running
> into a problem where emails that have been released from our outside
> spam protection company, *.protection.outlook.com, are getting
> rejected with messages like this:
>
>   Mar 26 06:00:56 mailhost postfix/smtpd[2270]: connect from mail-sn1nam01lp0113.outbound.protection.outlook.com[207.46.163.113]
>   Mar 26 06:00:56 mailhost postfix/smtpd[2270]: 51235A07D1: client=mail-sn1nam01lp0113.outbound.protection.outlook.com[207.46.163.113]
>   Mar 26 06:00:56 mailhost postfix/cleanup[2279]: 51235A07D1: message-id=<[hidden email]>
>   Mar 26 06:00:56 mailhost postfix/qmgr[27442]: 51235A07D1: from=<[hidden email]>, size=40439, nrcpt=1 (queue active)
>   Mar 26 06:00:56 mailhost postfix/local[2278]: 51235A07D1: to=<[hidden email]>, relay=local, delay=0.29, delays=0.28/0/0/0.01, dsn=5.4.6, status=bounced (mail forwarding loop for [hidden email])
>   Mar 26 06:00:56 mailhost postfix/bounce[2273]: 51235A07D1: sender non-delivery notification: 97DF2A080B
>   Mar 26 06:00:56 mailhost postfix/qmgr[27442]: 51235A07D1: removed
>
> These emails are released by the end user and should be delivered, but are getting bounced back.
>
> How would I go about figuring out if it's really a bogus "Delivered-To: " header that's causing this rejection?  
>

Some things you can do...

- search your logs for the message-id recorded above to see if this
message has been here before. Maybe this mail arrived before, was
forwarded off-site, then came back; don't do that.

- You can use the HOLD action to freeze an incoming message in the
queue before the local delivery agent has a chance to bounce it.
Then you can examine the message.   To HOLD the message, you can use
a check_recipient_access map, or a check_client_access map, or a
header_checks rule.

(NOTE: don't be tempted to use header_checks IGNORE to remove a
bogus Delivered-To header! The internet will thank you.)



  -- Noel Jones


>
>
> # postconf -n
> alias_database = hash:/etc/aliases
> alias_maps = nis:mail.aliases
> command_directory = /usr/sbin
> config_directory = /etc/postfix
> daemon_directory = /usr/libexec/postfix
> data_directory = /var/lib/postfix
> debug_peer_level = 2
> fallback_transport =
> html_directory = no
> inet_interfaces = all
> inet_protocols = ipv4
> local_header_rewrite_clients = static:all
> local_recipient_maps =
> mail_owner = postfix
> mailq_path = /usr/bin/mailq.postfix
> manpage_directory = /usr/share/man
> masquerade_domains = !hqmta.sub.com $myorigin
> message_size_limit = 30240000
> mydestination = $myhostname, localhost.$mydomain, localhost,
> $mydomain, sub.com, acs.sub.corp.com
> mydomain = sub.corp.com
> myhostname = mailhost.sub.corp.com
> mynetworks = 127.0.0.0/8, 209.243.0.0/16, 10.0.0.0/8
> myorigin = $mydomain
> newaliases_path = /usr/bin/newaliases.postfix
> queue_directory = /var/spool/postfix
> readme_directory = /usr/share/doc/postfix-2.6.6/README_FILES
> relay_domains = $mydestination, other.com, otherfoobar.com
> sample_directory = /usr/share/doc/postfix-2.6.6/samples
> sender_canonical_maps = hash:/etc/postfix/sender_canonical
> sendmail_path = /usr/sbin/sendmail.postfix
> setgid_group = postdrop
> transport_maps = hash:/etc/postfix/transport_maps
> unknown_local_recipient_reject_code = 450
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: problem with protection.outlook.com released spam getting bounced

John Stoffel-2
In reply to this post by Wietse Venema
>>>>> "Wietse" == Wietse Venema <[hidden email]> writes:

Wietse> Postfix reports this error because it is responsible for 'example.com'
Wietse> and the message has 'Delivered-To: [hidden email]'.

Thank you for your reply!  And thank you for postfix in general, it's
made my life simpler in so many ways.

Wietse> There are several options:

Wietse> 1) Your error. Your system is sending mail out after adding the
Wietse> 'Delivered-To: [hidden email]' header, and it comes back from
Wietse> outlook.com. If Postfix did not block such email then it could loop
Wietse> forever.

In this case no, the email is all coming from external people and
getting blocked by the spam filtering system.  When the user goes to
release it, then it gets looped back.

Wietse> 2) User error. After your system adds the 'Delivered-To: [hidden email]'
Wietse> header, some of your users forward their email off-site.  That email
Wietse> ends up at a system that looks at the message header address and
Wietse> that forwards that email to outlook.com. This results in a similar
Wietse> problem as (1). If Postfix did not block such email then it could
Wietse> loop forever.

This is something to check, but I'm not sure.

Wietse> 2) Outlook error. Outlook.com adds the 'Delivered-To: [hidden email]'
Wietse> header. In that case all mail from Outlook.com would have this
Wietse> problem. It does not seem likely.

Wietse> My money is on (1) or (2).

Is there any way I can keep a copy of these emails for debugging, even
though I still bounce them back?  I'd like to confirm what the headers
are if at all possible.

John


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: problem with protection.outlook.com released spam getting bounced

John Stoffel-2
In reply to this post by Noel Jones-2
>>>>> "Noel" == Noel Jones <[hidden email]> writes:

Noel> On 3/30/2017 9:26 AM, John Stoffel wrote:

>>
>> Hi all,
>>
>> We're running postfix-2.6.6-6.el6_5.x86_64 on RHEL 6.6 and running
>> into a problem where emails that have been released from our outside
>> spam protection company, *.protection.outlook.com, are getting
>> rejected with messages like this:
>>
>> Mar 26 06:00:56 mailhost postfix/smtpd[2270]: connect from mail-sn1nam01lp0113.outbound.protection.outlook.com[207.46.163.113]
>> Mar 26 06:00:56 mailhost postfix/smtpd[2270]: 51235A07D1: client=mail-sn1nam01lp0113.outbound.protection.outlook.com[207.46.163.113]
>> Mar 26 06:00:56 mailhost postfix/cleanup[2279]: 51235A07D1: message-id=<[hidden email]>
>> Mar 26 06:00:56 mailhost postfix/qmgr[27442]: 51235A07D1: from=<[hidden email]>, size=40439, nrcpt=1 (queue active)
>> Mar 26 06:00:56 mailhost postfix/local[2278]: 51235A07D1: to=<[hidden email]>, relay=local, delay=0.29, delays=0.28/0/0/0.01, dsn=5.4.6, status=bounced (mail forwarding loop for [hidden email])
>> Mar 26 06:00:56 mailhost postfix/bounce[2273]: 51235A07D1: sender non-delivery notification: 97DF2A080B
>> Mar 26 06:00:56 mailhost postfix/qmgr[27442]: 51235A07D1: removed
>>
>> These emails are released by the end user and should be delivered, but are getting bounced back.
>>
>> How would I go about figuring out if it's really a bogus "Delivered-To: " header that's causing this rejection?  
>>

Noel> Some things you can do...

Noel> - search your logs for the message-id recorded above to see if this
Noel> message has been here before. Maybe this mail arrived before, was
Noel> forwarded off-site, then came back; don't do that.

So I looked back through my logs until early February and I didn't see
it.  So it's not that sort of loop as far as I can tell.  

Noel> - You can use the HOLD action to freeze an incoming message in the
Noel> queue before the local delivery agent has a chance to bounce it.
Noel> Then you can examine the message.   To HOLD the message, you can use
Noel> a check_recipient_access map, or a check_client_access map, or a
Noel> header_checks rule.

Ok, thanks for the hints!  I'll have to read up on how to do a
header_checks rule and implement it so that I can see what's going on
here.  

Noel> (NOTE: don't be tempted to use header_checks IGNORE to remove a
Noel> bogus Delivered-To header! The internet will thank you.)

I know, but ... I might be forced to, since my users are bitching
about losing email they release from spam.  But!  I can also take this
to the vendor as proof they are doing something wrong as well.  But
first I need to get some messages and headers to look at first.

Thanks for your help Noel.

Noel>   -- Noel Jones


>>
>>
>> # postconf -n
>> alias_database = hash:/etc/aliases
>> alias_maps = nis:mail.aliases
>> command_directory = /usr/sbin
>> config_directory = /etc/postfix
>> daemon_directory = /usr/libexec/postfix
>> data_directory = /var/lib/postfix
>> debug_peer_level = 2
>> fallback_transport =
>> html_directory = no
>> inet_interfaces = all
>> inet_protocols = ipv4
>> local_header_rewrite_clients = static:all
>> local_recipient_maps =
>> mail_owner = postfix
>> mailq_path = /usr/bin/mailq.postfix
>> manpage_directory = /usr/share/man
>> masquerade_domains = !hqmta.sub.com $myorigin
>> message_size_limit = 30240000
>> mydestination = $myhostname, localhost.$mydomain, localhost,
>> $mydomain, sub.com, acs.sub.corp.com
>> mydomain = sub.corp.com
>> myhostname = mailhost.sub.corp.com
>> mynetworks = 127.0.0.0/8, 209.243.0.0/16, 10.0.0.0/8
>> myorigin = $mydomain
>> newaliases_path = /usr/bin/newaliases.postfix
>> queue_directory = /var/spool/postfix
>> readme_directory = /usr/share/doc/postfix-2.6.6/README_FILES
>> relay_domains = $mydestination, other.com, otherfoobar.com
>> sample_directory = /usr/share/doc/postfix-2.6.6/samples
>> sender_canonical_maps = hash:/etc/postfix/sender_canonical
>> sendmail_path = /usr/sbin/sendmail.postfix
>> setgid_group = postdrop
>> transport_maps = hash:/etc/postfix/transport_maps
>> unknown_local_recipient_reject_code = 450
>>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: problem with protection.outlook.com released spam getting bounced

John Stoffel-2
>>>>> "John" == John Stoffel <[hidden email]> writes:

>>>>> "Noel" == Noel Jones <[hidden email]> writes:
Noel> On 3/30/2017 9:26 AM, John Stoffel wrote:

>>>
>>> Hi all,
>>>
>>> We're running postfix-2.6.6-6.el6_5.x86_64 on RHEL 6.6 and running
>>> into a problem where emails that have been released from our outside
>>> spam protection company, *.protection.outlook.com, are getting
>>> rejected with messages like this:
>>>
>>> Mar 26 06:00:56 mailhost postfix/smtpd[2270]: connect from mail-sn1nam01lp0113.outbound.protection.outlook.com[207.46.163.113]
>>> Mar 26 06:00:56 mailhost postfix/smtpd[2270]: 51235A07D1: client=mail-sn1nam01lp0113.outbound.protection.outlook.com[207.46.163.113]
>>> Mar 26 06:00:56 mailhost postfix/cleanup[2279]: 51235A07D1: message-id=<[hidden email]>
>>> Mar 26 06:00:56 mailhost postfix/qmgr[27442]: 51235A07D1: from=<[hidden email]>, size=40439, nrcpt=1 (queue active)
>>> Mar 26 06:00:56 mailhost postfix/local[2278]: 51235A07D1: to=<[hidden email]>, relay=local, delay=0.29, delays=0.28/0/0/0.01, dsn=5.4.6, status=bounced (mail forwarding loop for [hidden email])
>>> Mar 26 06:00:56 mailhost postfix/bounce[2273]: 51235A07D1: sender non-delivery notification: 97DF2A080B
>>> Mar 26 06:00:56 mailhost postfix/qmgr[27442]: 51235A07D1: removed
>>>
>>> These emails are released by the end user and should be delivered, but are getting bounced back.
>>>
>>> How would I go about figuring out if it's really a bogus "Delivered-To: " header that's causing this rejection?  
>>>

Noel> Some things you can do...

Noel> - search your logs for the message-id recorded above to see if this
Noel> message has been here before. Maybe this mail arrived before, was
Noel> forwarded off-site, then came back; don't do that.

John> So I looked back through my logs until early February and I didn't see
John> it.  So it's not that sort of loop as far as I can tell.  

Noel> - You can use the HOLD action to freeze an incoming message in the
Noel> queue before the local delivery agent has a chance to bounce it.
Noel> Then you can examine the message.   To HOLD the message, you can use
Noel> a check_recipient_access map, or a check_client_access map, or a
Noel> header_checks rule.

John> Ok, thanks for the hints!  I'll have to read up on how to do a
John> header_checks rule and implement it so that I can see what's going on
John> here.  

Noel> (NOTE: don't be tempted to use header_checks IGNORE to remove a
Noel> bogus Delivered-To header! The internet will thank you.)

John> I know, but ... I might be forced to, since my users are bitching
John> about losing email they release from spam.  But!  I can also take this
John> to the vendor as proof they are doing something wrong as well.  But
John> first I need to get some messages and headers to look at first.

So I created the following entry in my header_checks file:

  /^Delivered-To:/ WARN Found email with Delivered-To: header already in it!

And while it did correctly warn on a bogus email that matched with
looping, it also matched on a bunch of other emails, which didn't get
rejected for looping.

So I'm wondering if the problem is that I'm not accepting email for my
legacy "foo.com" domain properly, while I am accepting email for my
"foo.bar.com" domain that we're now supposed to be using for all
email.

I'm glad I just did a warning match at first, instead of holding all
these emails, because it would have been a disaster for a bit until I
figured it out.  Is there anyway, besides the hold queue to just log
all the headers of these messages so I can try to understand the issue
in more detail?  I suspect that part of the problem is that we use
this server for outgoing emails, but all incoming from the internet
arrives through *.outbound.protection.outlook.com, so maybe they do
something to the headers?

My logs show the following:

  Mar 31 09:34:10 mailhost postfix/smtpd[28317]: connect from mail-sn1nam01lp0113.outbound.protection.
  outlook.com[207.46.163.113]
  Mar 31 09:34:10 mailhost postfix/smtpd[28317]: 7B6D1A05FE: client=mail-sn1nam01lp0113.outbound.prote
  ction.outlook.com[207.46.163.113]
  Mar 31 09:34:10 mailhost postfix/cleanup[28211]: 7B6D1A05FE: warning: header Delivered-To: [hidden email] from mail-sn1nam01lp0113.outbound.protection.outlook.com[207.46.163.113]; from=<[hidden email]> to=<[hidden email]> proto=ESMTP helo=<NAM01-SN1-obe.outbound.protection.outlook.com>: Found email with Delivered-To: header already in it!
  Mar 31 09:34:10 mailhost postfix/cleanup[28211]: 7B6D1A05FE: message-id=<[hidden email]>
  Mar 31 09:34:10 mailhost postfix/qmgr[27314]: 7B6D1A05FE: from=<[hidden email]>, size=62180, nrcpt=1 (queue active)
  Mar 31 09:34:10 mailhost postfix/local[28017]: 7B6D1A05FE: to=<[hidden email]>, relay=local, delay=0.33, delays=0.28/0/0/0.05, dsn=5.4.6, status=bounced (mail forwarding loop for [hidden email])
  Mar 31 09:34:10 mailhost postfix/bounce[28363]: 7B6D1A05FE: sender non-delivery notification: CCEC5A074E
  Mar 31 09:34:10 mailhost postfix/qmgr[27314]: 7B6D1A05FE: removed


So I'm matching things... but I'm also matching on alot of other emails which the logs look like this:


    Mar 31 09:36:21 mailhost postfix/smtpd[28317]: connect from hdqmta.foo.com[192.168.172.13]
    Mar 31 09:36:21 mailhost postfix/smtpd[28317]: E08F2A07A4: client=hdqmta.foo.com[192.168.172.13]
    Mar 31 09:36:21 mailhost postfix/cleanup[28191]: E08F2A07A4: warning: header Delivered-To: [hidden email] from hdqmta.foo.com[192.168.172.13]; from=<[hidden email]> to=<[hidden email]> proto=ESMTP helo=<hdqmta.foo.bar.com>: Found email with Delivered-To: header already in it!
    Mar 31 09:36:21 mailhost postfix/cleanup[28191]: E08F2A07A4: message-id=<[hidden email]>
    Mar 31 09:36:21 mailhost postfix/qmgr[27314]: E08F2A07A4: from=<[hidden email]>, size=7014, nrcpt=1 (queue active)
    Mar 31 09:36:21 mailhost postfix/smtpd[28317]: disconnect from hdqmta.foo.com[192.168.172.13]
    Mar 31 09:36:22 mailhost postfix/smtp[28312]: E08F2A07A4: to=<[hidden email]>, relay=smtp.na.bar.local[192.168.64.152]:25, delay=0.19, delays=0.01/0/0/0.18, dsn=2.6.0, status=sent (250 2.6.0 <[hidden email]> [InternalId=91027536871548, Hostname=NA-EXMB-P20.NA.BAR.LOCAL] Queued mail for delivery)
    Mar 31 09:36:22 mailhost postfix/qmgr[27314]: E08F2A07A4: removed

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: problem with protection.outlook.com released spam getting bounced

Noel Jones-2
On 3/31/2017 3:50 PM, John Stoffel wrote:
> So I created the following entry in my header_checks file:
>
>   /^Delivered-To:/ WARN Found email with Delivered-To: header already in it!
>
> And while it did correctly warn on a bogus email that matched with
> looping, it also matched on a bunch of other emails, which didn't get
> rejected for looping.

Postfix bounces mail when the Delivered-To address is the same as
the current recipient.


>
> So I'm wondering if the problem is that I'm not accepting email for my
> legacy "foo.com" domain properly, while I am accepting email for my
> "foo.bar.com" domain that we're now supposed to be using for all
> email.
>

The easiest explanation is that you are accepting mail for the old
domain, then forwarding it through outlook back to the new domain.
Or something similar.

> I'm glad I just did a warning match at first, instead of holding all
> these emails, because it would have been a disaster for a bit until I
> figured it out.  Is there anyway, besides the hold queue to just log
> all the headers of these messages so I can try to understand the issue
> in more detail?

To log all headers, use a header_check like:
/./  info

NOTE: postfix only allows one header_check rule per header, so this
will disable any header_checks below it.  Usually people put a
log-all rule like this at the end of header_checks file.

> I suspect that part of the problem is that we use
> this server for outgoing emails, but all incoming from the internet
> arrives through *.outbound.protection.outlook.com, so maybe they do
> something to the headers?

Test; don't speculate.  As a bystander, my job it to speculate: I
don't think outlook is adding the offending headers.



  -- Noel Jones

>
> My logs show the following:
>
>   Mar 31 09:34:10 mailhost postfix/smtpd[28317]: connect from mail-sn1nam01lp0113.outbound.protection.
>   outlook.com[207.46.163.113]
>   Mar 31 09:34:10 mailhost postfix/smtpd[28317]: 7B6D1A05FE: client=mail-sn1nam01lp0113.outbound.prote
>   ction.outlook.com[207.46.163.113]
>   Mar 31 09:34:10 mailhost postfix/cleanup[28211]: 7B6D1A05FE: warning: header Delivered-To: [hidden email] from mail-sn1nam01lp0113.outbound.protection.outlook.com[207.46.163.113]; from=<[hidden email]> to=<[hidden email]> proto=ESMTP helo=<NAM01-SN1-obe.outbound.protection.outlook.com>: Found email with Delivered-To: header already in it!
>   Mar 31 09:34:10 mailhost postfix/cleanup[28211]: 7B6D1A05FE: message-id=<[hidden email]>
>   Mar 31 09:34:10 mailhost postfix/qmgr[27314]: 7B6D1A05FE: from=<[hidden email]>, size=62180, nrcpt=1 (queue active)
>   Mar 31 09:34:10 mailhost postfix/local[28017]: 7B6D1A05FE: to=<[hidden email]>, relay=local, delay=0.33, delays=0.28/0/0/0.05, dsn=5.4.6, status=bounced (mail forwarding loop for [hidden email])
>   Mar 31 09:34:10 mailhost postfix/bounce[28363]: 7B6D1A05FE: sender non-delivery notification: CCEC5A074E
>   Mar 31 09:34:10 mailhost postfix/qmgr[27314]: 7B6D1A05FE: removed
>
>
> So I'm matching things... but I'm also matching on alot of other emails which the logs look like this:
>
>
>     Mar 31 09:36:21 mailhost postfix/smtpd[28317]: connect from hdqmta.foo.com[192.168.172.13]
>     Mar 31 09:36:21 mailhost postfix/smtpd[28317]: E08F2A07A4: client=hdqmta.foo.com[192.168.172.13]
>     Mar 31 09:36:21 mailhost postfix/cleanup[28191]: E08F2A07A4: warning: header Delivered-To: [hidden email] from hdqmta.foo.com[192.168.172.13]; from=<[hidden email]> to=<[hidden email]> proto=ESMTP helo=<hdqmta.foo.bar.com>: Found email with Delivered-To: header already in it!
>     Mar 31 09:36:21 mailhost postfix/cleanup[28191]: E08F2A07A4: message-id=<[hidden email]>
>     Mar 31 09:36:21 mailhost postfix/qmgr[27314]: E08F2A07A4: from=<[hidden email]>, size=7014, nrcpt=1 (queue active)
>     Mar 31 09:36:21 mailhost postfix/smtpd[28317]: disconnect from hdqmta.foo.com[192.168.172.13]
>     Mar 31 09:36:22 mailhost postfix/smtp[28312]: E08F2A07A4: to=<[hidden email]>, relay=smtp.na.bar.local[192.168.64.152]:25, delay=0.19, delays=0.01/0/0/0.18, dsn=2.6.0, status=sent (250 2.6.0 <[hidden email]> [InternalId=91027536871548, Hostname=NA-EXMB-P20.NA.BAR.LOCAL] Queued mail for delivery)
>     Mar 31 09:36:22 mailhost postfix/qmgr[27314]: E08F2A07A4: removed
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: problem with protection.outlook.com released spam getting bounced

John Stoffel-2
>>>>> "Noel" == Noel Jones <[hidden email]> writes:

Noel> On 3/31/2017 3:50 PM, John Stoffel wrote:
>> So I created the following entry in my header_checks file:
>>
>> /^Delivered-To:/ WARN Found email with Delivered-To: header already in it!
>>
>> And while it did correctly warn on a bogus email that matched with
>> looping, it also matched on a bunch of other emails, which didn't get
>> rejected for looping.

Noel> Postfix bounces mail when the Delivered-To address is the same as
Noel> the current recipient.

Yup, that's what I'm seeing here.  I upgraded to postfix 2.11.6 on
RHEL6 by hand, which went ok with just the minor bobble of needed to
add in the line:

   smtp_host_lookup = dns, native

because 'make upgrade' didn't do that.  

>> So I'm wondering if the problem is that I'm not accepting email for my
>> legacy "foo.com" domain properly, while I am accepting email for my
>> "foo.bar.com" domain that we're now supposed to be using for all
>> email.
>>

Noel> The easiest explanation is that you are accepting mail for the
Noel> old domain, then forwarding it through outlook back to the new
Noel> domain.  Or something similar.

I'm not.  Once I did my upgrade, I was able to add the line you
suggested to header_checks and I confirmed that the email released
from the EOP (Microsoft Spam filtering setup we use) only has received
headers from the sender of the spam, the mail.protection.outlook.com
servers, and then my server where it gets rejected with the mail
forwarding loop.  



>> I'm glad I just did a warning match at first, instead of holding all
>> these emails, because it would have been a disaster for a bit until I
>> figured it out.  Is there anyway, besides the hold queue to just log
>> all the headers of these messages so I can try to understand the issue
>> in more detail?

Noel> To log all headers, use a header_check like:
Noel> /./  info

Needed to upgrade from 2.6.6 (RHEL6) to 2.11.6 to make this work.

Noel> NOTE: postfix only allows one header_check rule per header, so this
Noel> will disable any header_checks below it.  Usually people put a
Noel> log-all rule like this at the end of header_checks file.

>> I suspect that part of the problem is that we use
>> this server for outgoing emails, but all incoming from the internet
>> arrives through *.outbound.protection.outlook.com, so maybe they do
>> something to the headers?

Noel> Test; don't speculate.  As a bystander, my job it to speculate: I
Noel> don't think outlook is adding the offending headers.

Someone is adding this header and it's not me from what I can see.

The incoming email DOES have a bunch of local NIS aliases:

   jojo -> happy -> happy@exchange-internal

But I would think that a Delivered-To: header doesn't get added until
the email passes all the way through the system onto the next
destination?

Sorry to be such a pain here, I'm pulling my hair out trying to chase
this down.  It all used to work before I put postfix in place to
replace an ancient crappy sendmail install.

John
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: problem with protection.outlook.com released spam getting bounced

John Stoffel-2

Well, I've confirmed that EOP (protection.outloko.com, our external
Spam filter provider) is adding in the "Delivered-To:" head when
emails that have been quarrantined are released to be delivered in to
us.

I'm amazed others haven't seen this problem yet, but maybe we're
strange.  In any case, now I need to figure out a way to fix this.

Would it be enough to simply remove the header if it arrives from
them?  I know it's a bad idea... but my customers are complaining
about this.

So a simple header_check like this might be what I want:

/^Delivered-To: (.*$)/ REPLACE EOP-Delivered-To: "$1"

But I only want this replamcent to happen for email that comes from a
specific set of outside servers.  I think I might have to run my own
milter here to do this.   I really can't depend on the headers not
being forged somehow, but I can depend on the host which connects to
me being who it says.

To a degree.

Am I making sense?
John
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: problem with protection.outlook.com released spam getting bounced

Viktor Dukhovni

> On Apr 4, 2017, at 5:26 PM, John Stoffel <[hidden email]> wrote:
>
> But I only want this replamcent to happen for email that comes from a
> specific set of outside servers.  I think I might have to run my own
> milter here to do this.   I really can't depend on the headers not
> being forged somehow, but I can depend on the host which connects to
> me being who it says.
>
> To a degree.
>
> Am I making sense?

Yes.  Another option is to build a custom version of Postfix in which
the Delivered-To: header is renamed.  Perhaps this should be configurable,
and maybe even forgery-protected, but an obscure alternative header would
suffice, if you're willing to run Postfix binaries built on site.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: problem with protection.outlook.com released spam getting bounced

Mike Guelfi
In reply to this post by John Stoffel-2
Assuming the header check works, I'd run that on a different instance  
of postfix and route the specific outside servers to that instance via  
the firewall...

Quoting John Stoffel <[hidden email]>:

> Well, I've confirmed that EOP (protection.outloko.com, our external
> Spam filter provider) is adding in the "Delivered-To:" head when
> emails that have been quarrantined are released to be delivered in to
> us.
>
> I'm amazed others haven't seen this problem yet, but maybe we're
> strange.  In any case, now I need to figure out a way to fix this.
>
> Would it be enough to simply remove the header if it arrives from
> them?  I know it's a bad idea... but my customers are complaining
> about this.
>
> So a simple header_check like this might be what I want:
>
> /^Delivered-To: (.*$)/ REPLACE EOP-Delivered-To: "$1"
>
> But I only want this replamcent to happen for email that comes from a
> specific set of outside servers.  I think I might have to run my own
> milter here to do this.   I really can't depend on the headers not
> being forged somehow, but I can depend on the host which connects to
> me being who it says.
>
> To a degree.
>
> Am I making sense?
> John


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: problem with protection.outlook.com released spam getting bounced

John Stoffel-2

They handle all our incoming email... so I might instead do something
where I handle internal email on a seperate setup.  What a pain...

Mike> Assuming the header check works, I'd run that on a different
Mike> instance of postfix and route the specific outside servers to
Mike> that instance via the firewall...

Mike> Quoting John Stoffel <[hidden email]>:

>> Well, I've confirmed that EOP (protection.outloko.com, our external
>> Spam filter provider) is adding in the "Delivered-To:" head when
>> emails that have been quarrantined are released to be delivered in to
>> us.
>>
>> I'm amazed others haven't seen this problem yet, but maybe we're
>> strange.  In any case, now I need to figure out a way to fix this.
>>
>> Would it be enough to simply remove the header if it arrives from
>> them?  I know it's a bad idea... but my customers are complaining
>> about this.
>>
>> So a simple header_check like this might be what I want:
>>
>> /^Delivered-To: (.*$)/ REPLACE EOP-Delivered-To: "$1"
>>
>> But I only want this replamcent to happen for email that comes from a
>> specific set of outside servers.  I think I might have to run my own
>> milter here to do this.   I really can't depend on the headers not
>> being forged somehow, but I can depend on the host which connects to
>> me being who it says.
>>
>> To a degree.
>>
>> Am I making sense?
>> John


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: problem with protection.outlook.com released spam getting bounced

John Stoffel-2
In reply to this post by Viktor Dukhovni
>>>>> "Viktor" == Viktor Dukhovni <[hidden email]> writes:

>> On Apr 4, 2017, at 5:26 PM, John Stoffel <[hidden email]> wrote:
>>
>> But I only want this replamcent to happen for email that comes from a
>> specific set of outside servers.  I think I might have to run my own
>> milter here to do this.   I really can't depend on the headers not
>> being forged somehow, but I can depend on the host which connects to
>> me being who it says.
>>
>> To a degree.
>>
>> Am I making sense?

Viktor> Yes.  Another option is to build a custom version of Postfix in which
Viktor> the Delivered-To: header is renamed.  Perhaps this should be configurable,
Viktor> and maybe even forgery-protected, but an obscure alternative header would
Viktor> suffice, if you're willing to run Postfix binaries built on site.

Since I built 2.11.9 by hand, I'm willing to do this hack as well I
think.  It's a total hack too... and I'm still amazed I'm the only one
seeing this.  But maybe most people who use Office365 for spam
filtering also use Exchange on the inside and not the setup we have
which is due to Lotus Notes, Mailman and Exchange all being valid
internal targets.  Whee...

Looks like I just need to hack src/global/mail_copy.c and
src/global/delivered_hrd.c, or better yet, just change the
global/header_opts.c to maybe be something like this:

diff -ur postfix-2.11.9/src/global/header_opts.c postfix-2.11.9.jfs/src/global/header_opts.c
--- postfix-2.11.9/src/global/header_opts.c     2017-04-10 12:50:34.381884494 -0700
+++ postfix-2.11.9.jfs/src/global/header_opts.c 2008-05-08 13:41:35.000000000 -0700
@@ -62,7 +62,7 @@
     "Content-Length", HDR_CONTENT_LENGTH, HDR_OPT_DROP,
     "Content-Transfer-Encoding", HDR_CONTENT_TRANSFER_ENCODING, HDR_OPT_MIME,
     "Content-Type", HDR_CONTENT_TYPE, HDR_OPT_MIME,
-    "Delivered-To", HDR_DELIVERED_TO, 0,
+    "My-Delivered-To", HDR_DELIVERED_TO, 0,
     "Disposition-Notification-To", HDR_DISP_NOTIFICATION, HDR_OPT_SENDER,
     "Date", HDR_DATE, 0,
     "Errors-To", HDR_ERRORS_TO, HDR_OPT_SENDER,


Which certainly can't hurt as a test.

John
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: problem with protection.outlook.com released spam getting bounced

Viktor Dukhovni

> On Apr 10, 2017, at 4:01 PM, John Stoffel <[hidden email]> wrote:
>
> Since I built 2.11.9 by hand, I'm willing to do this hack as well I
> think.  It's a total hack too... and I'm still amazed I'm the only one
> seeing this.  But maybe most people who use Office365 for spam
> filtering also use Exchange on the inside and not the setup we have
> which is due to Lotus Notes, Mailman and Exchange all being valid
> internal targets.  Whee...
>
> Looks like I just need to hack src/global/mail_copy.c and
> src/global/delivered_hrd.c, or better yet, just change the
> global/header_opts.c to maybe be something like this:
>
> diff -ur postfix-2.11.9/src/global/header_opts.c postfix-2.11.9.jfs/src/global/header_opts.c
> --- postfix-2.11.9/src/global/header_opts.c     2017-04-10 12:50:34.381884494 -0700
> +++ postfix-2.11.9.jfs/src/global/header_opts.c 2008-05-08 13:41:35.000000000 -0700
> @@ -62,7 +62,7 @@
>     "Content-Length", HDR_CONTENT_LENGTH, HDR_OPT_DROP,
>     "Content-Transfer-Encoding", HDR_CONTENT_TRANSFER_ENCODING, HDR_OPT_MIME,
>     "Content-Type", HDR_CONTENT_TYPE, HDR_OPT_MIME,
> -    "Delivered-To", HDR_DELIVERED_TO, 0,
> +    "My-Delivered-To", HDR_DELIVERED_TO, 0,
>     "Disposition-Notification-To", HDR_DISP_NOTIFICATION, HDR_OPT_SENDER,
>     "Date", HDR_DATE, 0,
>     "Errors-To", HDR_ERRORS_TO, HDR_OPT_SENDER,
>
>
> Which certainly can't hurt as a test.

All three need to be changed to a consistent replacement:

src/global/header_opts.c:    "Delivered-To", HDR_DELIVERED_TO, 0,
src/global/mail_copy.c: vstream_fprintf(dst, "Delivered-To: %s%s", vstring_str(buf), eol);
src/local/forward.c:    rec_fprintf(info->cleanup, REC_TYPE_NORM, "Delivered-To: %s",

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: problem with protection.outlook.com released spam getting bounced

John Stoffel-2
>>>>> "Viktor" == Viktor Dukhovni <[hidden email]> writes:

>> On Apr 10, 2017, at 4:01 PM, John Stoffel <[hidden email]> wrote:
>>
>> Since I built 2.11.9 by hand, I'm willing to do this hack as well I
>> think.  It's a total hack too... and I'm still amazed I'm the only one
>> seeing this.  But maybe most people who use Office365 for spam
>> filtering also use Exchange on the inside and not the setup we have
>> which is due to Lotus Notes, Mailman and Exchange all being valid
>> internal targets.  Whee...
>>
>> Looks like I just need to hack src/global/mail_copy.c and
>> src/global/delivered_hrd.c, or better yet, just change the
>> global/header_opts.c to maybe be something like this:
>>
>> diff -ur postfix-2.11.9/src/global/header_opts.c postfix-2.11.9.jfs/src/global/header_opts.c
>> --- postfix-2.11.9/src/global/header_opts.c     2017-04-10 12:50:34.381884494 -0700
>> +++ postfix-2.11.9.jfs/src/global/header_opts.c 2008-05-08 13:41:35.000000000 -0700
>> @@ -62,7 +62,7 @@
>> "Content-Length", HDR_CONTENT_LENGTH, HDR_OPT_DROP,
>> "Content-Transfer-Encoding", HDR_CONTENT_TRANSFER_ENCODING, HDR_OPT_MIME,
>> "Content-Type", HDR_CONTENT_TYPE, HDR_OPT_MIME,
>> -    "Delivered-To", HDR_DELIVERED_TO, 0,
>> +    "My-Delivered-To", HDR_DELIVERED_TO, 0,
>> "Disposition-Notification-To", HDR_DISP_NOTIFICATION, HDR_OPT_SENDER,
>> "Date", HDR_DATE, 0,
>> "Errors-To", HDR_ERRORS_TO, HDR_OPT_SENDER,
>>
>>
>> Which certainly can't hurt as a test.

Viktor> All three need to be changed to a consistent replacement:

Viktor> src/global/header_opts.c:    "Delivered-To", HDR_DELIVERED_TO, 0,
Viktor> src/global/mail_copy.c: vstream_fprintf(dst, "Delivered-To: %s%s", vstring_str(buf), eol);
Viktor> src/local/forward.c:    rec_fprintf(info->cleanup, REC_TYPE_NORM, "Delivered-To: %s",

Well I did this and put it into production and it seems to be working
just fine.  I can now get at least one test email released from
quarrantine and have it get delivered properly.

I've also got a ticket opened with Outlook.com, but I don't hold alot
of faith in them getting back to me.  A big thank you to Viktor for
his help, and of course to Wietse for writing this in the first
place.  I shudder to think what it would have taken to do this in
Sendmail...

John
Loading...