Transport based on next hop

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Transport based on next hop

Christian Rößner
Hi,

I have a trivial question, which could become a wish list feature.

There are three MTAs. First is a web server postfix instance that relates all mail to the second MTAS, a relay server, which can send mail directly to the world. This relay server and a third MTA are two postfix multi instances.

The relay server is for all kinds of other satellites (other machines with i.e. Cron and log heck messages), for some business customers, who want to send newsletters or mails with a little bit larger attachments. Short: the relay server may become in trouble, if bad recipient addresses are in place,

The third instance is the real MTA. All received mail is coming in on this instance, as normal mail from submission is sent to the world. This server also does SPF, DKIM, DMARC on incoming mail.

Now the problem that came up this day:

A customer has a website the probably has a contact form. Internally it sends mail with sendmail command, setting the envelope address to [hidden email]. Unfortunately the form does set a From:-header to an AOL address.

The email destination is managed by the real MTA. So what happened? Mail arrived on www with sendmail, was delivered to the relay server that contacted the real MTA and this server rejected the mail with DMARC policy.

Fully correct behavior.

Now my question:

When the relay server determines the MX for a recipient address, is there any table that works like this:

MX host is foo bar, use transport SMTP:[some.mta]:12345

So I could define a SMTP-in for the relay server, if the real MTA is responsible for the destination. I could disable all kinds of millers and make the relay server a mynetworks only hook.

Thanks in advance

Christian
Reply | Threaded
Open this post in threaded view
|

Re: Transport based on next hop

A. Schulze

Christen Rößner:

> Unfortunately the form does set a From:-header to an AOL address.
(ask the customer to) fix that

> MX host is foo bar, use transport SMTP:[some.mta]:12345
There was as similar discussion on this list some days ago:
   http://marc.info/?t=141668781400001

configure domain based routing. it's what just work out of the box.

Andreas


Reply | Threaded
Open this post in threaded view
|

Re: Transport based on next hop

Wietse Venema
In reply to this post by Christian Rößner
Christen R??ner:
> When the relay server determines the MX for a recipient address, is there any table that works like this:
>
> MX host is foo bar, use transport SMTP:[some.mta]:12345

To override MX host selection based on recipient:
http://www.postfix.org/postconf.5.html#transport_maps

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Transport based on next hop

Christian Rößner


> Am 28.11.2014 um 18:05 schrieb Wietse Venema <[hidden email]>:
>
> Christen R??ner:
>> When the relay server determines the MX for a recipient address, is there any table that works like this:
>>
>> MX host is foo bar, use transport SMTP:[some.mta]:12345
>
> To override MX host selection based on recipient:
> http://www.postfix.org/postconf.5.html#transport_maps

Sorry, I guess this not help.

If I read your documentation correctly, the transport_maps parameter does select a next hop based on a recipient address. That is not a solution here.

I look for:

Table:
Lhs                        Rhs
mx.some.mta        smtp:[mx.whatever.tld]:1234

With this, mail would not be routed to mx.some.mta:25, I could define a separate master.cf entry

I even could totally overwrite a MX record.

Example:

Recipient address lookup for [hidden email] would result in a MX result of mx.deltaweb.de. And now I look for a way to say: do not send to mx.deltaweb.de:25, but consult a lookup table for this MX, which could give me another port. Or even another MTA.

Christian

>
>    Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Transport based on next hop

Christian Rößner

> Am 28.11.2014 um 19:26 schrieb Christen Rößner <[hidden email]>:
>
> Recipient address lookup for [hidden email] would result in a MX result of mx.deltaweb.de. And now I look for a way to say: do not send to mx.deltaweb.de:25, but consult a lookup table for this MX, which could give me another port. Or even another MTA.

The example is buggy, as my typing auto correction broke the example email. But you still can get the idea.
Reply | Threaded
Open this post in threaded view
|

Re: Transport based on next hop

Noel Jones-2
In reply to this post by Christian Rößner
On 11/28/2014 12:26 PM, Christen Rößner wrote:

>
>
>> Am 28.11.2014 um 18:05 schrieb Wietse Venema <[hidden email]>:
>>
>> Christen R??ner:
>>> When the relay server determines the MX for a recipient address, is there any table that works like this:
>>>
>>> MX host is foo bar, use transport SMTP:[some.mta]:12345
>>
>> To override MX host selection based on recipient:
>> http://www.postfix.org/postconf.5.html#transport_maps
>
> Sorry, I guess this not help.
>
> If I read your documentation correctly, the transport_maps parameter does select a next hop based on a recipient address. That is not a solution here.
>
> I look for:
>
> Table:
> Lhs                        Rhs
> mx.some.mta        smtp:[mx.whatever.tld]:1234
>
> With this, mail would not be routed to mx.some.mta:25, I could define a separate master.cf entry
>
> I even could totally overwrite a MX record.
>

All postfix transport decisions are based on the recipient domain,
not the MX it resolves to.

It should be possible to use a TCP or socketmap transport table to
override the recipient MX.  I don't know of any existing code for
this, so you would need to write something yourself.


  -- Noel Joness
Reply | Threaded
Open this post in threaded view
|

Re: Transport based on next hop

lists@rhsoft.net
In reply to this post by Christian Rößner


Am 28.11.2014 um 19:31 schrieb Christen Rößner:
>> Am 28.11.2014 um 19:26 schrieb Christen Rößner <[hidden email]>:
>>
>> Recipient address lookup for [hidden email] would result in a MX result of mx.deltaweb.de. And now I look for a way to say: do not send to mx.deltaweb.de:25, but consult a lookup table for this MX, which could give me another port. Or even another MTA.
>
> The example is buggy, as my typing auto correction broke the example email. But you still can get the idea.

i get what you want to achieve

*but* wouldn't be the way to go to have a sender-dependent relayhost on
that MX instead try to hack the lookups? if you don't control the MX or
not can work together with the admin on the other side it's questionable
at all

the 4 options below on "mx.deltaweb.de" would achive that inclduing port
and athentication

if you want to bypass the MX use [hostname]

* sender_dependent_relayhost_maps
* smtp_sasl_password_maps
* smtp_sasl_auth_enable
* smtp_sender_dependent_authentication
Reply | Threaded
Open this post in threaded view
|

Re: Transport based on next hop

Christian Rößner


> Am 28.11.2014 um 19:39 schrieb "[hidden email]" <[hidden email]>:
>
>
>
> Am 28.11.2014 um 19:31 schrieb Christen Rößner:
>>> Am 28.11.2014 um 19:26 schrieb Christen Rößner <[hidden email]>:
>>>
>>> Recipient address lookup for [hidden email] would result in a MX result of mx.deltaweb.de. And now I look for a way to say: do not send to mx.deltaweb.de:25, but consult a lookup table for this MX, which could give me another port. Or even another MTA.
>>
>> The example is buggy, as my typing auto correction broke the example email. But you still can get the idea.
>
> i get what you want to achieve
>
> *but* wouldn't be the way to go to have a sender-dependent relayhost on that MX instead try to hack the lookups? if you don't control the MX or not can work together with the admin on the other side it's questionable at all

You are right and on the other side, I do control both servers. It's a postmulti setup. So two instances on the same host.

A dirty hack would be DNATing with iptables, but I really would prefer clean routing.

And as I do not have full control over the www server, running Postfix, I can not influence users behavior. It's a hosting server with hundreds of users.

So controlling based on MX next hop overwrite could be much easier and cleaner.

> the 4 options below on "mx.deltaweb.de" would achive that inclduing port and athentication
>
> if you want to bypass the MX use [hostname]
>
> * sender_dependent_relayhost_maps
> * smtp_sasl_password_maps
> * smtp_sasl_auth_enable
> * smtp_sender_dependent_authentication

I will look at these, even I think a solution might be difficult, if it is sender based.

Christian
Reply | Threaded
Open this post in threaded view
|

Re: Transport based on next hop

Wietse Venema
In reply to this post by Christian Rößner
Christen R??ner:
> I look for:
>
> Table:
> Lhs                        Rhs
> mx.some.mta        smtp:[mx.whatever.tld]:1234

I have implemented smtp_dns_reply_filter (currently, testing), which
matches a resource record against a (regular expression) pattern.

At the moment supports IGNORE, but it would not be difficult to
implement a REPLACE action. The content of the resource record have
been "valid-hostname" sanity checked, so there is little danger of
malicious substitutions.

Thus, Postfix could do

foo.example.com. 12345 IN MX 10 bar.example.com.
        REPLACE foo.example.com. 12345 IN MX 10 baz.example.net.

Similar operations for A/AAAA records,with some limitations (can
change the last two fields only, not the fields that are determined
by the query itself).

Would that solve the problem?

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Transport based on next hop

Christian Rößner


> Am 28.11.2014 um 20:26 schrieb Wietse Venema <[hidden email]>:
>
> Christen R??ner:
>> I look for:
>>
>> Table:
>> Lhs                        Rhs
>> mx.some.mta        smtp:[mx.whatever.tld]:1234
>
> I have implemented smtp_dns_reply_filter (currently, testing), which
> matches a resource record against a (regular expression) pattern.
>
> At the moment supports IGNORE, but it would not be difficult to
> implement a REPLACE action. The content of the resource record have
> been "valid-hostname" sanity checked, so there is little danger of
> malicious substitutions.
>
> Thus, Postfix could do
>
> foo.example.com. 12345 IN MX 10 bar.example.com.
>    REPLACE foo.example.com. 12345 IN MX 10 baz.example.net.
>
> Similar operations for A/AAAA records,with some limitations (can
> change the last two fields only, not the fields that are determined
> by the query itself).
>
> Would that solve the problem?
How could a different port be specified?  If I do understand your example, this only replaces the host, but gives no choice to specify a port

Christian

>
>    Wietse

smime.p7s (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Transport based on next hop

lists@rhsoft.net


Am 28.11.2014 um 20:45 schrieb Christian Rößner:

>> Am 28.11.2014 um 20:26 schrieb Wietse Venema <[hidden email]>:
>>
>> Christen R??ner:
>>> I look for:
>>>
>>> Table:
>>> Lhs                        Rhs
>>> mx.some.mta        smtp:[mx.whatever.tld]:1234
>>
>> I have implemented smtp_dns_reply_filter (currently, testing), which
>> matches a resource record against a (regular expression) pattern.
>>
>> At the moment supports IGNORE, but it would not be difficult to
>> implement a REPLACE action. The content of the resource record have
>> been "valid-hostname" sanity checked, so there is little danger of
>> malicious substitutions.
>>
>> Thus, Postfix could do
>>
>> foo.example.com. 12345 IN MX 10 bar.example.com.
>>     REPLACE foo.example.com. 12345 IN MX 10 baz.example.net.
>>
>> Similar operations for A/AAAA records,with some limitations (can
>> change the last two fields only, not the fields that are determined
>> by the query itself).
>>
>> Would that solve the problem?
>
> How could a different port be specified?  If I do understand your example, this only replaces the host, but gives no choice to specify a port

MTA-to-MTA traffic is typically port 25

if you need several instances on the same machine talking with each
other consider giving that machine additional IP addresses with their
own DNS names and bind the instance on the IP instead mangle around
with differnt ports

the benefit is that mail routing in that case don't need port-hacks and
so can be realized with hostnames in a agnostic way
Reply | Threaded
Open this post in threaded view
|

Re: Transport based on next hop

Wietse Venema
In reply to this post by Christian Rößner
> >> Table:
> >> Lhs                        Rhs
> >> mx.some.mta        smtp:[mx.whatever.tld]:1234

If "mx.some.mta" was found via the recipient, domain, consider
using a pattern based on the recipient.

If "mx.some.mta" was found via the sender, consider using a
sender-based acces rule with

    sender-pattern      filter smtp:[mx.whatever.tld]:1234

instead.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Transport based on next hop

Christian Rößner
In reply to this post by lists@rhsoft.net

> Am 28.11.2014 um 20:50 schrieb "[hidden email]" <[hidden email]>:
>
>
>
> Am 28.11.2014 um 20:45 schrieb Christian Rößner:
>>> Am 28.11.2014 um 20:26 schrieb Wietse Venema <[hidden email]>:
>>>
>>> Christen R??ner:
>>>> I look for:
>>>>
>>>> Table:
>>>> Lhs                        Rhs
>>>> mx.some.mta        smtp:[mx.whatever.tld]:1234
>>>
>>> I have implemented smtp_dns_reply_filter (currently, testing), which
>>> matches a resource record against a (regular expression) pattern.
>>>
>>> At the moment supports IGNORE, but it would not be difficult to
>>> implement a REPLACE action. The content of the resource record have
>>> been "valid-hostname" sanity checked, so there is little danger of
>>> malicious substitutions.
>>>
>>> Thus, Postfix could do
>>>
>>> foo.example.com. 12345 IN MX 10 bar.example.com.
>>>    REPLACE foo.example.com. 12345 IN MX 10 baz.example.net.
>>>
>>> Similar operations for A/AAAA records,with some limitations (can
>>> change the last two fields only, not the fields that are determined
>>> by the query itself).
>>>
>>> Would that solve the problem?
>>
>> How could a different port be specified?  If I do understand your example, this only replaces the host, but gives no choice to specify a port
>
> MTA-to-MTA traffic is typically port 25
>
> if you need several instances on the same machine talking with each other consider giving that machine additional IP addresses with their own DNS names and bind the instance on the IP instead mangle around
> with differnt ports
>
> the benefit is that mail routing in that case don't need port-hacks and so can be realized with hostnames in a agnostic way
This server already has two ip addresses and routing can not be done on answer decisions. That exactly is the problem here.

And the main MTA on port 25 enforces a policy. It doesn't matter, if mail is coming from foreign MTAs or from the relay server on the same host. But it needs a different policy, if the relay server is contacting the main MTA. And this can only be achieved with a different port on the main MAT, as I only can turn off Milters and things in a separate instance.

If I add an instance on the main MTA on i.e. port 12325, I can set smtpd_milters empty and mynerworks to the IP of the relay server, restricting smtpd_recipient_restrictions to permit_mynetworks,reject.

I can enforce a replacement with:

iptables -t nat -A PREROUTING -j DNAT -s ip.of.relay -d ip.of.main.mta -p tcp --dport 25 --to-destination ip.of.main.mta:12325

That is, what I would like to solve cleanly.

Christian

smime.p7s (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Transport based on next hop

A. Schulze

Christian Rößner:

> This server already has two ip addresses and routing can not be done  
> on answer decisions. That exactly is the problem here.
>
> And the main MTA on port 25 enforces a policy.

As you told in a previus message you run multiple instances on one host.
I assume you have a clean setup about which instance use which ip address.
-> inet_interfaces = ${myhostname}

In that case I suggest to setup a secondary address on the loopback interface.
The "main MTA" could listen there with a separate smtp instance  
(without policy)
and the "relay MTA" could use that IP replaced by the DNS filter  
Wietse mentioned.

Andreas

Reply | Threaded
Open this post in threaded view
|

SOLVED Re: Transport based on next hop

Christian Rößner
>> This server already has two ip addresses and routing can not be done on answer decisions. That exactly is the problem here.

>>
>> And the main MTA on port 25 enforces a policy.
>
> As you told in a previus message you run multiple instances on one host.
> I assume you have a clean setup about which instance use which ip address.
> -> inet_interfaces = ${myhostname}
>
> In that case I suggest to setup a secondary address on the loopback interface.
> The "main MTA" could listen there with a separate smtp instance (without policy)
> and the "relay MTA" could use that IP replaced by the DNS filter Wietse mentioned.
After a call with Robert and one night of sleep, I found a good solution.

My former policy was, to check for SPF, DKIM, DMARC only on MX-in. Now I also added these policies to submission and the relay server and it works. With these new policies, a customer can not inject mail with bogus headers. This also prevents my system from sending spam from infected websites that have modified the sender addresses. I love this solution. The downside is that mail delivery is slightly slower. But I think this is okay.

Fighting problems at the source.

Thanks for all of you

Christian
--
Bachelor of Science Informatik
Erlenwiese 14, 36304 Alsfeld
T: +49 6631 78823400, F: +49 6631 78823409, M: +49 171 9905345
USt-IdNr.: DE225643613, http://www.roessner-network-solutions.com


smime.p7s (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: SOLVED Re: Transport based on next hop

Stephen Satchell
On 11/29/2014 03:52 AM, Christian Rößner wrote:
> The downside is that mail delivery is slightly
> slower. But I think this is okay.

Mail is designed to do everything it can to ensure delivery.  Delivery
speed is secondary.
Reply | Threaded
Open this post in threaded view
|

Re: Transport based on next hop

Christian Rößner
In reply to this post by Christian Rößner
Hi,

> I have a trivial question, which could become a wish list feature.
>
> There are three MTAs. First is a web server postfix instance that relates all mail to the second MTAS, a relay server, which can send mail directly to the world. This relay server and a third MTA are two postfix multi instances.
>
> The relay server is for all kinds of other satellites (other machines with i.e. Cron and log heck messages), for some business customers, who want to send newsletters or mails with a little bit larger attachments. Short: the relay server may become in trouble, if bad recipient addresses are in place,
>
> The third instance is the real MTA. All received mail is coming in on this instance, as normal mail from submission is sent to the world. This server also does SPF, DKIM, DMARC on incoming mail.
>
> Now the problem that came up this day:
>
> A customer has a website the probably has a contact form. Internally it sends mail with sendmail command, setting the envelope address to [hidden email]. Unfortunately the form does set a From:-header to an AOL address.
>
> The email destination is managed by the real MTA. So what happened? Mail arrived on www with sendmail, was delivered to the relay server that contacted the real MTA and this server rejected the mail with DMARC policy.
>
> Fully correct behavior.
>
> Now my question:
>
> When the relay server determines the MX for a recipient address, is there any table that works like this:
>
> MX host is foo bar, use transport SMTP:[some.mta]:12345
>
> So I could define a SMTP-in for the relay server, if the real MTA is responsible for the destination. I could disable all kinds of millers and make the relay server a mynetworks only hook.
I took me some time, but I finally wrote a milter vrfydmn that can handle such problems. And some mire… :-)

If you are interested in the milter, feel free to have a look at

https://github.com/croessner/vrfydmn

It still needs more documentation, but a first README.md is provided and also a picture of my current mail system that demonstrates the use of this milter.

Feedback welcome

Christian
--
Bachelor of Science Informatik
Erlenwiese 14, 36304 Alsfeld
T: +49 6631 78823400, F: +49 6631 78823409, M: +49 171 9905345
USt-IdNr.: DE225643613, http://www.roessner-network-solutions.com


signature.asc (506 bytes) Download Attachment