relay_transport backup/secondary

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

relay_transport backup/secondary

lutz.niederer
Hi,
 
we are using two external MX servers in separate data centers.  Both of them are running postfix since many years without problems.
 
Internally we do have a postfix server as final destination for all domains.  On each MX we have defined a relay_transport with specific settings that relay mail to our internal server.  For this we use transport:[A RECORD]:port as relay_transport.
 
Now we have two links to our internal server and we observed that it was the right decision to have those two links.  Our internal server got two IP addresses (one per link) and is listening on both of them for connections from the MX servers.  But we have no clue how to set up a secondary relay_transport that kicks in if the primary is offline.  So, currently we cannot use the backup link for incoming mails.  We can only specify one relay_transport entry, means one IP address, with a transport.
 
Maybe smtp_fallback_relay would be a solution but we are unsure.  smtp_fallback_relay got the problem that it seems to use the smtp transport only, and then we are unable to set all the transport stuff that we need (header manipulation, certificates, filters, etc).
We also thought that we could use transport:MX RECORD:port and give them different prios using DNS but in our special case this is not possible.
What is the right way to define a secondary relay_transport that kicks in if the primary is down and switches back to the primary if it comes up again?  (We would like to change as few as possible because this is a pretty well running system.)
 
I thank the community for tips, hints and help!
 
Cheers!
-lutzn
 
 
 
Reply | Threaded
Open this post in threaded view
|

Re: relay_transport backup/secondary

Viktor Dukhovni


> On Jan 14, 2018, at 8:53 AM, [hidden email] wrote:
>
> we are using two external MX servers in separate data centers.  Both of them are running postfix since many years without problems.
>  
> Internally we do have a postfix server as final destination for all domains.  On each MX we have defined a relay_transport with specific settings that relay mail to our internal server.  For this we use transport:[A RECORD]:port as relay_transport.
>  
> Now we have two links to our internal server and we observed that it was the right decision to have those two links.  Our internal server got two IP addresses (one per link) and is listening on both of them for connections from the MX servers.  But we have no clue how to set up a secondary relay_transport that kicks in if the primary is offline.  So, currently we cannot use the backup link for incoming mails.  We can only specify one relay_transport entry, means one IP address, with a transport.

Replace the IP address with a hostname:

   main.cf:
        indexed = ${default_database_type}:${config_directory}/
        transport_maps = ${indexed}transport
        relay_host_lookup = dns, native

   transport:
        example.com relay:[internal.example.com]

   master.cf:
        relay unix ... smtp
          -o smtp_host_lookup=$relay_host_lookup

   /etc/hosts
        # See http://www.linfo.org/etc_host_conf.html
        192.0.2.1 internal.example.com
        192.0.2.2 internal.example.com

This will deliver mail via either address chosen uniformly randomly.
If you need a fallback use MX records:

   main.cf:
        indexed = ${default_database_type}:${config_directory}/
        transport_maps = ${indexed}transport

   transport:
        example.com relay:example.com.localhost

and configure the local DNS resolver to return MX records for
example.com.localhost:

        ; localhost private zone
        localhost. IN A 127.0.0.1
        example.com.localhost. IN MX 10 relay1.example.com.localhost.
        example.com.localhost. IN MX 20 relay2.example.com.localhost.
        relay1.example.com.localhost. IN A 192.0.2.1
        relay2.example.com.localhost. IN A 192.0.2.2

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: relay_transport backup/secondary

Wietse Venema
In reply to this post by lutz.niederer
[hidden email]:
> Now we have two links to our internal server and we observed that
> it was the right decision to have those two links. Our internal
> server got two IP addresses (one per link) and is listening on
> both of them for connections from the MX servers. But we have no
> clue how to set up a secondary relay_transport that kicks in if
> the primary is offline. So, currently we cannot use the backup
> link for incoming mails. We can only specify one relay_transport
> entry, means one IP address, with a transport.

In the transport map, configure one of:

    transport:dns-name-with-multiple-mx-or-a-records:port
    transport:non-dns-name-with-multiple-ip-addresses:port

The first form allows the specification of preferences,
and the second form does not.

Another possibility is to specify the secondary relay with
smtp_fallback_relay.

> Maybe smtp_fallback_relay would be a solution but we are unsure.
> smtp_fallback_relay got the problem that it seems to use the smtp
> transport only, and then we are unable to set all the transport
> stuff that we need (header manipulation, certificates, filters,
> etc).

Indeed, smtp_fallback_relay is built into the SMTP client, so it
uses the same 'transport', i.e. it shares the same rate limits,
concurrency limits, etc.

Here's a straw-man design for multiple transport:nexthop results
in transport maps. Not sure if it solves the problem in this thread
but I'm writing it down just in case.

Support for multiple transport:nexthop results per recipient would
require minor changes in the trivial-rewrite resolver (to return
multiple transport:nexthop results per recipient), invasive changes
in the Postfix scheduler (to iterate over multiple transport:nexthop
results per recipient).

There also needs to be a hint from the scheduler to delivery agents
so that they don't log "status=deferred" for a recipient that still
has an unused transport:nexthop.

Limitations

After the first transport:nexthop was unable to deliver the recipient,
each attempt to use an alternative transport:nexthop puts that
recipient at the back of the in-memory queue for that transport and
nexthop. So there may be some scheduling delays between the first
transport:nexthop and later ones for the same recipient.

If the command "postfix reload" is invoked, the scheduler will not
attempt to use any unused transport:nexthop results. This means
that a delivery agent will not log "status=deferred", if it received
a hint from the scheduler because that recipient still had an unused
transport:nexthop.

Load balancing

A flat list of transport_maps results makes some things easier (to
specify a primary destination before secondary ones) and some things
harder (to distribute load evenly across equal-preference destinations).

Even distribution would require a feature to shuffle transport map
results, for example with special transport map syntax:

/etc/postfix/main.cf:
    transport_maps = hash:/etc/postfix/transport

/etc/postfix/transport
    example.com: shuffle:{transport1:nexthop1, transport2:nexthop2}

or a 'shuffle' pseudo table that randomizes the results from other
tables:

/etc/postfix/main.cf:
    tranport_maps = shuffle:hash:/etc/postfix/transport

/etc/postfix/transport
    example.com: transport1:nexthop1, transport2:nexthop2

I'd prefer the second solution as it is more general (it works
for everything that reads a list from a table).

        Wietse