question on fallback transport usage

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

question on fallback transport usage

pandorasbox55
Hi All -

Here's our scenario. 

Migrating from server A to server B. Server A is a proprietary system that uses LDAP queries for delivering email. Server B is a Postfix system configured to deliver email using LDAP queries, mirroring as closely as possible, Server A. 

In testing, the Postfix server appears to be processing a large majority of mail correctly but we keep running into 'edge cases' where Postfix fails on delivering an email that the original server is able to deliver successfully. We have tried tracking these edge case failures through the mail logs, and correcting them as we find them, but we still run into these cases here and there. 

What we would like to do, as a transitional state, is mimic the Sendmail failover relay to re-direct undeliverable email from the Postfix system to Server A. With Server A only handling emails redirected to it from the Postfix server, this would allow us to more easily identify the edge cases.

We think we could do this using fallback transport but before we go too deep, we'd like to see if anyone else has tried that and if it's a good idea or if anyone has a better way to tackle that scenario. In our scenario, we would only use the fallback transport until we completed migrating to the new system.

thanks in advance. 


Reply | Threaded
Open this post in threaded view
|

Re: question on fallback transport usage

Viktor Dukhovni


> On Dec 27, 2017, at 5:15 PM, l carr <[hidden email]> wrote:
>
> What we would like to do, as a transitional state, is mimic the Sendmail failover relay to re-direct undeliverable email from the Postfix system to Server A. With Server A only handling emails redirected to it from the Postfix server, this would allow us to more easily identify the edge cases.
>
> We think we could do this using fallback transport but before we go too deep, we'd like to see if anyone else has tried that and if it's a good idea or if anyone has a better way to tackle that scenario. In our scenario, we would only use the fallback transport until we completed migrating to the new system.

The "fallback_transport" setting is a feature of the local(8)
delivery agent, when it finds no alias or local user matching
the localpart of the email address.

If all the problem messages are for a domain in mydestination,
and fail to find a matching "alias" in local delivery, then
indeed fallback_transport might do the trick.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: question on fallback transport usage

pandorasbox55

The domains are not defined under mydestination, they are defined under virtual_alias_domains. So it sounds like the fallback_transport may not work for us. Is there any other way to accomplish that same scenario? 


=lc




From: [hidden email] <[hidden email]> on behalf of Viktor Dukhovni <[hidden email]>
Sent: Wednesday, December 27, 2017 5:31 PM
To: Postfix users
Subject: Re: question on fallback transport usage
 


> On Dec 27, 2017, at 5:15 PM, l carr <[hidden email]> wrote:
>
> What we would like to do, as a transitional state, is mimic the Sendmail failover relay to re-direct undeliverable email from the Postfix system to Server A. With Server A only handling emails redirected to it from the Postfix server, this would allow us to more easily identify the edge cases.
>
> We think we could do this using fallback transport but before we go too deep, we'd like to see if anyone else has tried that and if it's a good idea or if anyone has a better way to tackle that scenario. In our scenario, we would only use the fallback transport until we completed migrating to the new system.

The "fallback_transport" setting is a feature of the local(8)
delivery agent, when it finds no alias or local user matching
the localpart of the email address.

If all the problem messages are for a domain in mydestination,
and fail to find a matching "alias" in local delivery, then
indeed fallback_transport might do the trick.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: question on fallback transport usage

Wietse Venema
l carr:
> The domains are not defined under mydestination, they are defined
> under virtual_alias_domains. So it sounds like the fallback_transport
> may not work for us. Is there any other way to accomplish that
> same scenario?

That depends on what you mean with 'undeliverable'. Which Postfix
delivery agent logs the non-delivery, and what is logged as the
delivery status? You can anonymize the email address.

(Viktor and I are the main developers, so you're talking to the
right people).

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: question on fallback transport usage

Viktor Dukhovni
In reply to this post by pandorasbox55


> On Dec 27, 2017, at 8:42 PM, l carr <[hidden email]> wrote:
>
> The domains are not defined under mydestination, they are defined under
> virtual_alias_domains. So it sounds like the fallback_transport may not
> work for us. Is there any other way to accomplish that same scenario?

Just change move the domains from virtual_alias_domains to relay_domains:

 main.cf:
   indexed = ${default_database_type}:${config_directory}/
   parent_domain_matches_subdomains = smtpd_access_maps
   relay_domains = ldap-complete.example
   relay_transport = relay:[legacy-server.example]
   virtual_alias_domains = ldap-incomplete.example
   virtual_alias_maps = ldap:${config_directory}/ldap-valias.cf

   # If you have no list of valid recipients, as a last resort
   # accept all relay recipients
   #
   relay_recipient_maps = static:all

   # Otherwise deploy some suitable table that lists all valid
   # recipients.
   #
   # relay_recipient_maps = ...

This works because virtual alias domains always rewrite into some
underlying domain for delivery, which works already.  So any remaining
recipients that don't get rewritten can be handled via relay_transport
after changing the problem domains to relay_domains.  After all the
problem recipients are resolved, move them back to the virtual alias
domains list.

You might end up with a bit of a backscatter problem if you can't
enumerate valid recipients.  Don't let this fester.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: question on fallback transport usage

pandorasbox55
In reply to this post by Wietse Venema
I see that both you and Viktor responded to my posting, thank you. While Viktor provided a potential solution, I am answering your questions here in case this information is still relevant to the issue.


- To be 'undeliverable' means the entry exists in the LDAP but either the entry is configured in such a way the Postfix LDAP queries cannot determine how to deliver the email or there is an issue with the LDAP query itself. 99% of the time, it should be the LDAP entry and we should be able to correct the entry.
- The status is logged as "User unknown in virtual alias table"
- We did not have a currently broken example so we have recreated a previously broken account to get you the logs that you requested. We believe the error could happen within two different delivery agents.

    smtp - The error appears here when the email is transferred from the upstream server and fails immediately.

    local - The error appears here when the email is transferred from the upstream server but must 'walk' the LDAP to find the delivery information in another account entry. (I tried to replicate this failure using sendmail -bv on the server but not sure if that will provide you the log information you need.)

(The postfix server is currently set to softbounce while in testing mode.)

Failure Type #1
postfixServer postfix/smtpd[12152]: C4875A3A4D: client=orginatingServer
postfixServer postfix/cleanup[12450]: C4875A3A4D: message-id=<>
postfixServer postfix/qmgr[2440]: C4875A3A4D: from=<testuser@domain>, size=204, nrcpt=1 (queue active)
postfixServer postfix/error[12460]: C4875A3A4D: to=<testu33@domain>, relay=none, delay=15, delays=15/0.03/0/0.03, dsn=4.1.1, status=SOFTBOUNCE (User unknown in virtual alias table)

Failure Type #2 (Tested using sendmail -bv)

postfix/pickup[5930]: ACC90A3A4F: uid=0 from=<testuser>
postfix/cleanup[12696]: ACC90A3A4F: message-id=<[hidden email]>
postfix/qmgr[2440]: ACC90A3A4F: from=<testuser@domain>, size=273, nrcpt=1 (queue active)
postfix/error[12698]: ACC90A3A4F: to=<testu33@domain>, relay=none, delay=0.04, delays=0.03/0.01/0/0.01, dsn=5.1.1, status=undeliverable (User unknown in virtual alias table)
postfix/qmgr[2440]: ACC90A3A4F: removed


thanks again,
=lc




From: [hidden email] <[hidden email]> on behalf of Wietse Venema <[hidden email]>
Sent: Wednesday, December 27, 2017 9:20 PM
To: Postfix users
Subject: Re: question on fallback transport usage
 
l carr:
> The domains are not defined under mydestination, they are defined
> under virtual_alias_domains. So it sounds like the fallback_transport
> may not work for us. Is there any other way to accomplish that
> same scenario?

That depends on what you mean with 'undeliverable'. Which Postfix
delivery agent logs the non-delivery, and what is logged as the
delivery status? You can anonymize the email address.

(Viktor and I are the main developers, so you're talking to the
right people).

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: question on fallback transport usage

pandorasbox55
In reply to this post by Viktor Dukhovni

Thank you, Viktor. We will try your recommended configuration.


One question from your email:


- We're not sure what you mean by a list of valid recipients so I'll state - In our scenario, the Postfix server is an intermediary server, and not accessible from outside of our IP space. Mail that it processes will have already been screened and any emails it receives should be for valid recipients within the the domains it configured to process. It only needs to determine the final delivery point for those messages and deliver them. (Which is what is uses the LDAP queries for.) I believe, based on that, we will be using the static:all config option that you stated below. (This will be tested before trying with moving real mail.)


- No worries on letting this fester.. this is truly just a stop gap to find the edge case LDAP entries and correct them. We will only run this long enough to get a feel that we've caught any glaring issues. Then we'll return the Postfix configuration to the proper config and shutdown the old server.


=lc




From: [hidden email] <[hidden email]> on behalf of Viktor Dukhovni <[hidden email]>
Sent: Wednesday, December 27, 2017 11:00 PM
To: Postfix users
Subject: Re: question on fallback transport usage
 


> On Dec 27, 2017, at 8:42 PM, l carr <[hidden email]> wrote:
>
> The domains are not defined under mydestination, they are defined under
> virtual_alias_domains. So it sounds like the fallback_transport may not
> work for us. Is there any other way to accomplish that same scenario?

Just change move the domains from virtual_alias_domains to relay_domains:

 main.cf:
   indexed = ${default_database_type}:${config_directory}/
   parent_domain_matches_subdomains = smtpd_access_maps
   relay_domains = ldap-complete.example
   relay_transport = relay:[legacy-server.example]
   virtual_alias_domains = ldap-incomplete.example
   virtual_alias_maps = ldap:${config_directory}/ldap-valias.cf

   # If you have no list of valid recipients, as a last resort
   # accept all relay recipients
   #
   relay_recipient_maps = static:all

   # Otherwise deploy some suitable table that lists all valid
   # recipients.
   #
   # relay_recipient_maps = ...

This works because virtual alias domains always rewrite into some
underlying domain for delivery, which works already.  So any remaining
recipients that don't get rewritten can be handled via relay_transport
after changing the problem domains to relay_domains.  After all the
problem recipients are resolved, move them back to the virtual alias
domains list.

You might end up with a bit of a backscatter problem if you can't
enumerate valid recipients.  Don't let this fester.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: question on fallback transport usage

Viktor Dukhovni


> On Dec 29, 2017, at 1:18 PM, l carr <[hidden email]> wrote:
>
> One question from your email:
>
> - We're not sure what you mean by a list of valid recipients

A complete list of the email addresses that exist in the domain,
allowing you to definitively reject email messages addressed to
recipients that do not exist (and would bounce after forwarding
to the legacy server).

> In our scenario, the Postfix server is an intermediary server, and
> not accessible from outside of our IP space. Mail that it processes
> will have already been screened and any emails it receives should be
> for valid recipients within the the domains it configured to process.

In that case the lack of recipient validation is likely a minor issue.

> It only needs to determine the final delivery point for those messages
> and deliver them. (Which is what is uses the LDAP queries for.) I
> believe, based on that, we will be using the static:all config option
> that you stated below. (This will be tested before trying with moving
> real mail.)

Should work fine.

--
        Viktor.