Toss load-balancer health checks, but BCC everything else (always_bcc, check_sender_access and 'smtpd_delay_reject = yes')

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Toss load-balancer health checks, but BCC everything else (always_bcc, check_sender_access and 'smtpd_delay_reject = yes')

deoren
My apologies if I overlooked an answer somewhere, but I checked the docs
and performed a brief search of the archives before asking and didn't
spot the answer.

Goal:

BCC everything EXCEPT for health check emails generated by our HAProxy
load-balancer

I originally tried following the directions provided by Victor Duchovni
(see link below), but I evidently didn't follow along well enough to
replicate the results.

I seem to have found a combination of settings which accomplishes my
goal and have listed them below. Are there any issues that you can spot
with my approach?

always_bcc = [hidden email]
smtpd_delay_reject = yes
smtpd_recipient_restrictions =
   ...
   check_sender_access proxy:mysql:/etc/postfix/mysql-sender_access.cf,
   ...


Within the access table I'm mapping the health check sender address to
the DISCARD action. This appears to accomplish the original goal of
exercising the entire intake workflow (port is open, Postfix is running,
sender IP is allowed, sender email address is allowed, address
resolution is working) AND also prevents those health check emails from
going to the always_bcc email address while still sending everything
else over.

Thanks in advance for reading this and offering suggestions.

Refs:

http://www.postfix.org/postconf.5.html
http://www.postfix.org/access.5.html
http://www.postfix.org/pcre_table.5.html
http://postfix.1071664.n5.nabble.com/exclude-email-account-from-always-bcc-td13695.html

Reply | Threaded
Open this post in threaded view
|

Re: Toss load-balancer health checks, but BCC everything else (always_bcc, check_sender_access and 'smtpd_delay_reject = yes')

Wietse Venema
deoren:
> My apologies if I overlooked an answer somewhere, but I checked the docs
> and performed a brief search of the archives before asking and didn't
> spot the answer.
>
> Goal:
>
> BCC everything EXCEPT for health check emails generated by our HAProxy
> load-balancer

Use a CIDR map:

/etc/postfix/main.cf
    smtpd_client_restritions =
        check_client_access cidr:/etc/postfix/client-access.cidr

etc/postfix/client-access.cidr:
    # First, a rule that matches healtcheck mail.
    1.2.3.4/32 DUNNO
    # Add a BCC recipient to other email.
    0.0.0.0/0 BCC [hidden email]

This requires Postfix 3.0 or later.

        Wietse

Reply | Threaded
Open this post in threaded view
|

Re: Toss load-balancer health checks, but BCC everything else (always_bcc, check_sender_access and 'smtpd_delay_reject = yes')

@lbutlr
In reply to this post by deoren
On 11 May 2018, at 09:55, deoren <[hidden email]> wrote:
> BCC everything EXCEPT for health check emails generated by our HAProxy load-balancer

Seems it would be much simpler to BCC everything and then discard the few messages you don’t want.

--
I WILL NOT INSTIGATE REVOLUTION Bart chalkboard Ep. 7G06

Reply | Threaded
Open this post in threaded view
|

Re: Toss load-balancer health checks, but BCC everything else (always_bcc, check_sender_access and 'smtpd_delay_reject = yes')

deoren
In reply to this post by Wietse Venema
On 5/11/2018 2:20 PM, Wietse Venema wrote:

> deoren:
>> My apologies if I overlooked an answer somewhere, but I checked the docs
>> and performed a brief search of the archives before asking and didn't
>> spot the answer.
>>
>> Goal:
>>
>> BCC everything EXCEPT for health check emails generated by our HAProxy
>> load-balancer
>
> Use a CIDR map:
>
> /etc/postfix/main.cf
>      smtpd_client_restritions =
> check_client_access cidr:/etc/postfix/client-access.cidr
>
> etc/postfix/client-access.cidr:
>      # First, a rule that matches healtcheck mail.
>      1.2.3.4/32 DUNNO
>      # Add a BCC recipient to other email.
>      0.0.0.0/0 BCC [hidden email]
>
> This requires Postfix 3.0 or later.
>
> Wietse
>

Thank you for the feedback and the solution.

I assume that the performance for this solution is going to be much
better than the approach I went with? I'm guessing that this approach
will also be much more stable over the long run as well. To continue
that thought, this approach also keeps the two specific actions together
in a single block which should be easier for anyone reviewing the
settings to mentally parse.
Reply | Threaded
Open this post in threaded view
|

Re: Toss load-balancer health checks, but BCC everything else (always_bcc, check_sender_access and 'smtpd_delay_reject = yes')

deoren
In reply to this post by @lbutlr
On 5/12/2018 1:51 AM, @lbutlr wrote:
> On 11 May 2018, at 09:55, deoren <[hidden email]> wrote:
>> BCC everything EXCEPT for health check emails generated by our HAProxy load-balancer
>
> Seems it would be much simpler to BCC everything and then discard the few messages you don’t want.
>

Thanks for your feedback.

I thought of that, but with our load-balancer sending in an email to
each node every X seconds (which, admittedly might be overkill) that
would result in stream of emails going to the archive account each time
the health check runs. I was thinking that if there was an issue with
the relay nodes handing off their mail then those health checks would
clutter the queue and cause delays for legitimate mail.

I was hoping with some logic in place to block the health check emails
from going through the standard intake "flow" (I know I'm using the
right terms here) that other mail wouldn't be held up if there was a
problem delivering those emails.
Reply | Threaded
Open this post in threaded view
|

Re: Toss load-balancer health checks, but BCC everything else (always_bcc, check_sender_access and 'smtpd_delay_reject = yes')

deoren
In reply to this post by deoren
On 5/13/2018 11:56 AM, deoren wrote:

> On 5/11/2018 2:20 PM, Wietse Venema wrote:
>> deoren:
>>> My apologies if I overlooked an answer somewhere, but I checked the docs
>>> and performed a brief search of the archives before asking and didn't
>>> spot the answer.
>>>
>>> Goal:
>>>
>>> BCC everything EXCEPT for health check emails generated by our HAProxy
>>> load-balancer
>>
>> Use a CIDR map:
>>
>> /etc/postfix/main.cf
>>      smtpd_client_restritions =
>>     check_client_access cidr:/etc/postfix/client-access.cidr
>>
>> etc/postfix/client-access.cidr:
>>      # First, a rule that matches healtcheck mail.
>>      1.2.3.4/32 DUNNO
>>      # Add a BCC recipient to other email.
>>      0.0.0.0/0 BCC [hidden email]
>>
>> This requires Postfix 3.0 or later.
>>
>>     Wietse
>>
>
> Thank you for the feedback and the solution.
>
> I assume that the performance for this solution is going to be much
> better than the approach I went with? I'm guessing that this approach
> will also be much more stable over the long run as well. To continue
> that thought, this approach also keeps the two specific actions together
> in a single block which should be easier for anyone reviewing the
> settings to mentally parse.

I had a few moments to go back and update the setup to use the proposed
solution, but stopped part way through. I see that we're matching on the
client IP here with an action of DUNNO.

Instead of matching on the IP with check_client_access, could we match
on the sender address instead with check_sender_access so that
non-health-check mail from the load-balancer will still be subject to
the BCC action?

  /etc/postfix/main.cf
       smtpd_client_restrictions =
         check_sender_access hash:/etc/postfix/sender-access.cf

  /etc/postfix/sender-access.cf:
       # First, a rule that matches health-check mail.
       [hidden email] DUNNO
       # Add a BCC recipient to other email.
       * BCC [hidden email]

For the last entry that BCC's "everything else", is the * character a
valid source?

Thanks for your help.
Reply | Threaded
Open this post in threaded view
|

Re: Toss load-balancer health checks, but BCC everything else (always_bcc, check_sender_access and 'smtpd_delay_reject = yes')

Wietse Venema
deoren:
>   /etc/postfix/sender-access.cf:
>        # First, a rule that matches health-check mail.
>        [hidden email] DUNNO
>        # Add a BCC recipient to other email.
>        * BCC [hidden email]
>
> For the last entry that BCC's "everything else", is the * character a
> valid source?

The access map syntax is documented. http://www.postfix.org/access.5.html
There is no '*' in there.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Toss load-balancer health checks, but BCC everything else (always_bcc, check_sender_access and 'smtpd_delay_reject = yes')

Wietse Venema
Wietse Venema:

> deoren:
> >   /etc/postfix/sender-access.cf:
> >        # First, a rule that matches health-check mail.
> >        [hidden email] DUNNO
> >        # Add a BCC recipient to other email.
> >        * BCC [hidden email]
> >
> > For the last entry that BCC's "everything else", is the * character a
> > valid source?
>
> The access map syntax is documented. http://www.postfix.org/access.5.html
> There is no '*' in there.

If you must filter on sender address, you can use a regexp: or pcre:
table.

smtpd_xxx_restrictions =
    ...
    check_sender_access pcre:/etc/postfix/sender-access.pcre
    ...

/etc/postfix/sender-access.pcre:
    # First, a rule that matches health-check mail.
    /^smtp-health-checks@example\.com$/ DUNNO
    # Add a BCC recipient to other email.
    . BCC [hidden email]

The ^, $, \, and . are special only when you use regexp: or pcre:.
They are not part of the access map syntax.

Instead of the above pair of rules you could use if..endif, but
again, those are not part of the access map syntax, but of the
specific lookup table.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Toss load-balancer health checks, but BCC everything else (always_bcc, check_sender_access and 'smtpd_delay_reject = yes')

deoren
On 5/17/2018 3:20 PM, Wietse Venema wrote:

> Wietse Venema:
>> deoren:
>>>    /etc/postfix/sender-access.cf:
>>>         # First, a rule that matches health-check mail.
>>>         [hidden email] DUNNO
>>>         # Add a BCC recipient to other email.
>>>         * BCC [hidden email]
>>>
>>> For the last entry that BCC's "everything else", is the * character a
>>> valid source?
>>
>> The access map syntax is documented. http://www.postfix.org/access.5.html
>> There is no '*' in there.
>
> If you must filter on sender address, you can use a regexp: or pcre:
> table.
>
> smtpd_xxx_restrictions =
>      ...
>      check_sender_access pcre:/etc/postfix/sender-access.pcre
>      ...
>
> /etc/postfix/sender-access.pcre:
>      # First, a rule that matches health-check mail.
>      /^smtp-health-checks@example\.com$/ DUNNO
>      # Add a BCC recipient to other email.
>      . BCC [hidden email]
>
> The ^, $, \, and . are special only when you use regexp: or pcre:.
> They are not part of the access map syntax.
>
> Instead of the above pair of rules you could use if..endif, but
> again, those are not part of the access map syntax, but of the
> specific lookup table.
>
> Wietse
>

Thank you for your help. Re the '*' syntax, I wasn't sure how to
accomplish the goal of matching everything, so thanks for clearing that
up. I ended up going with this setup based off of your feedback and the
original example I found from Victor Duchovni:

/etc/postfix/main.cf
     # BCC mail flowing through this relay EXCEPT for
     # health check emails for future troubleshooting
     pcre:/etc/postfix/auto_bcc.pcre

     # Other checks/actions
     proxy:mysql:/etc/postfix/mysql-sender_access.cf,


/etc/postfix/auto_bcc.pcre
     # though this content is indented to stand out
     # there is no indention in the file itself
     if !/^haproxy-smtp-check@example\.com$/
     /@/ BCC [hidden email]
     endif


So far it appears to be working just fine. Thanks again for your help.