Central filtering postfix relay

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Central filtering postfix relay

Phil Ingram
Hello,

We are using postfix as a central email relay that forwards to an external provider for trusted sending to our customers. Centralising this relay is a must to limit the distribution of sasl creds required for sending to our external provider. We have several products, each with dev, staging and production environments and each with their own defined Class A address ranges (10.0.0.0/16). Every server has an FQDN which makes email sent to accounts such as 'root' from CRON easy to filter - "To: root@$host.app-environment.local".

From reading through the header_checks doc this seems like the best place to do conditional filtering, but, per the doc, you can not filter on From: To: Message-ID: Date:; the former two are of most use. It looks like I can filter on ^Received:.*10\.0\.\d{1,3}\.\d{1,3}, but this is not enough.

For brevity's sake in the following I am omitting the actual regex, but from quite a lot of testing and using 'postmap -q - pcre:/etc/postfix/header_checks </tmp/test-email' I can see that the rule set "should" work..

What we wish to achieve (with pcre because it's more efficient):


if /^Received:.*PRODUCT_IP_RANGE_REGEX/
/^Received:.*DEV_IP_REGEX/ REDIRECT [hidden email]
/^Received:.*STAGING_IP_REGEX/ REDIRECT [hidden email]
# /PRODUCTION_IP_REGEX/ should be permitted to be sent to our external provider
# Other rules for a specific product that we do not wish to apply to other products should also go here.
endif

/^To:.*\.local$/ REDIRECT devnull@localhost.
/^To:.*\.internal$/ REDIRECT devnull@localhost.
/^Received:/ IGNORE


But:
1) As soon as a redirect is actioned the "To: local" at the bottom doesn't work; but the "To: local" does catch and redirect email that was not matched and redirected inside of the "production_ip_range" if/endif
2) If i put the "To: local" filter at the top, it does REDIRECT, but then the IP range regex overrides it, which I see is the intended behaviour noted in the docs and why we put the .local catch at the bottom.
3) As soon as REDIRECT has been actioned once I can no longer match on ^To: or even just ".*To.*".


I have pretty much exhausted my Search Fu and am beginning to think that this is simply not possible. Is anyone able to confirm that I'm crazy and this is indeed not possible, or is there something I am missing?


Regards,


Phil
Reply | Threaded
Open this post in threaded view
|

Re: Central filtering postfix relay

Viktor Dukhovni


> On May 3, 2018, at 8:37 PM, Phil Ingram <[hidden email]> wrote:
>
> We are using postfix as a central email relay that forwards to an external provider for trusted sending to our customers. Centralising this relay is a must to limit the distribution of sasl creds required for sending to our external provider. We have several products, each with dev, staging and production environments and each with their own defined Class A address ranges (10.0.0.0/16). Every server has an FQDN which makes email sent to accounts such as 'root' from CRON easy to filter - "To: root@$host.app-environment.local".

There's your mistake, you're trying to route mail based or header regular
expressions.  Don't do that.  Route mail based on the envelope recipient
address, your address rewriting tables (aliases, virtual aliases, etc.) and
transport settings.  If needed, consider a separate null-client instance
for cron-generated mail.

But first explain in more detail where the mail that needs to go to the
provider originates from, and how it is different from the mail that
should not go to the provider.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: Central filtering postfix relay

Phil Ingram

There are a few different types of mail we deal with:

1. Cron email from things we don't care about and while these are mostly set as &>/dev/null anyway, but we need to ensure that we have a catch all for the 'just in case' scenario where someone makes a furfy (i.e. >&/dev/null).
2. Cron email from things we do care about. (reports, logwatch, maintenance tasks, etc.)
3. Application emails; password resets, new account registration, notifications etc. that are sent to our customers

Depending on which product the email is coming from we have a need to handle email differently between dev/stag/prod; i.e. permit production servers to send emails to our customers, but with staging/uat redirect everything to an internal dist list for developer review. We use Ammutable infrastructure (custom built AWS images that are partially configured post launch) and having to update each and every image then deploy it to make a small change is a larger overhead than we would like as it's easier for us to just 'relayhost' to a central postfix instance where we can more easily filter out and redirect the emails we care about.



Phil Ingram

Infrastructure & Security Manager
Community Data Solutions

214 Greenhill Rd
Eastwood, SA 5063
1800 503 981

On 4 May 2018 at 12:06, Viktor Dukhovni <[hidden email]> wrote:


> On May 3, 2018, at 8:37 PM, Phil Ingram <[hidden email]> wrote:
>
> We are using postfix as a central email relay that forwards to an external provider for trusted sending to our customers. Centralising this relay is a must to limit the distribution of sasl creds required for sending to our external provider. We have several products, each with dev, staging and production environments and each with their own defined Class A address ranges (10.0.0.0/16). Every server has an FQDN which makes email sent to accounts such as 'root' from CRON easy to filter - "To: root@$host.app-environment.local".

There's your mistake, you're trying to route mail based or header regular
expressions.  Don't do that.  Route mail based on the envelope recipient
address, your address rewriting tables (aliases, virtual aliases, etc.) and
transport settings.  If needed, consider a separate null-client instance
for cron-generated mail.

But first explain in more detail where the mail that needs to go to the
provider originates from, and how it is different from the mail that
should not go to the provider.

--
        Viktor.


Reply | Threaded
Open this post in threaded view
|

Re: Central filtering postfix relay

Viktor Dukhovni


> On May 3, 2018, at 11:13 PM, Phil Ingram <[hidden email]> wrote:
>
> 1. Cron email from things we don't care about and while these are mostly set as &>/dev/null anyway, but we need to ensure that we have a catch all for the 'just in case' scenario where someone makes a furfy (i.e. >&/dev/null).
> 2. Cron email from things we do care about. (reports, logwatch, maintenance tasks, etc.)
> 3. Application emails; password resets, new account registration, notifications etc. that are sent to our customers
>
> Depending on which product the email is coming from we have a need to handle email differently between dev/stag/prod; i.e. permit production servers to send emails to our customers, but with staging/uat redirect everything to an internal dist list for developer review. We use Ammutable infrastructure (custom built AWS images that are partially configured post launch) and having to update each and every image then deploy it to make a small change is a larger overhead than we would like as it's easier for us to just 'relayhost' to a central postfix instance where we can more easily filter out and redirect the emails we care about.

This is not actionable information.  You need to provide more detail, such
as e.g. someone would need to attempt to route each type of traffic to the
right place.

Also, though you might prefer to take care of everything on the mailhub,
if the trasition from the hosts to the mailhub loses critical differences
as to the type of mail being handled, then the correct solution is to
do appropriate rewriting at the sources, before delivery to the mailhub.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: Central filtering postfix relay

Phil Ingram

This is not actionable information.
I think I misinterpreted your original request. Mail that needs to go to the external provider originates from any number of hosts/servers on a number of different networks. Email sent from these servers originates from cron, other daemons and our application. So, to the central relay the sending hosts and the email they receive looks no different, but the To:, From: and Received: info in each email does.

(application/cron/daemon) -> server's local postfix -> central relay's postfix -> external provider -> mail.google -> end user

In the case of a UAT environment, to get the most consistent view of any changes made, server configuration, application configuration and data must be 1:1 with production. So, having a different configuration of each server's local postfix breaks this, which is why we would prefer to redirect application email from the UAT env before it leaves our network.
 
You need to provide more detail, such
as e.g. someone would need to attempt to route each type of traffic to the
right place.
I'm sorry, I don't follow.
 

Also, though you might prefer to take care of everything on the mailhub,
if the trasition from the hosts to the mailhub loses critical differences
as to the type of mail being handled
We don't care, all email is unclassified, we are tolerant of failures and completely control the private network. We would simply like to control the routing and redirection of email based on the source network and destination email address
 

Reply | Threaded
Open this post in threaded view
|

Re: Central filtering postfix relay

Viktor Dukhovni


> On May 4, 2018, at 1:04 AM, Phil Ingram <[hidden email]> wrote:
>
> We don't care, all email is unclassified, we are tolerant of failures and completely control the private network. We would simply like to control the routing and redirection of email based on the source network and destination email address

You can implement iptables rules on the mailhub to deliver traffic from
some clients to one Postfix instance with one set of transport tables, ...
and traffic from other clients to another.

When you say "source network", how literally do you mean that?

--
        Viktor.