Failing open with recipient address verification

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Failing open with recipient address verification

Jay Deiman
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello all,

Here is a large problem I've been struggling with for a while now.  I'm
managing some backup MX servers that, unfortunately, do not have lists
of valid recipient addresses for the domains we are backing up.  Of
course this means that they are a large source of backscatter, which is
not at all what I want.

Recently, I turned on recipient address verification and I have to say
that I'm quite impressed with it.  It's just about exactly what I was
looking for and solves all my issues, except for it's one downfall.

The problem, in terms of it's behavior, that I have is that I want it to
fail open (accept mail for a domain) when the smtp server that I'm
checking against for a valid recipient is not responding at all.
Unfortunately, this is not the case.  What I'm seeing is the email
request is denied with a 450 at the backup mx server when the primary mx
server is not responding.

Is there any way around this that anyone knows of?  I currently have
tried using a different smtp transport in master.cf with a fallback
relay set to a separate smtpd running on a different port on localhost
and that does not seem to work as I had hoped.  I was hoping that it
would fall back and verify at the local smtpd after the connection to
the primary MX, which should respond OK.  Here is the relevant parts of
my master.cf that I'm talking about:

smtp      inet  n       -       n       -       500      smtpd
127.0.0.1:10025 inet n  -       n       -       500      smtpd
~    -o
smtpd_recipient_restrictions=permit_mynetworks,reject_unauth_destination

smtpver   unix  -       -       n       -       -       smtp
~    -o smtp_fallback_relay=127.0.0.1:10025

In my address_verify_transport_maps:

domain.org      smtpver:

Thank you,

Jay Deiman
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIX9htQ0lr+ZVKSBgRAjzqAJ9lsxO+Sga9HBjz8Dysd1/N9eEJNgCfTbzK
3GP1lG5n9/rUXJZ6Y1eKixw=
=O2tM
-----END PGP SIGNATURE-----
Reply | Threaded
Open this post in threaded view
|

Re: Failing open with recipient address verification

Charles Marcus
On 6/23/2008, Jay Deiman ([hidden email]) wrote:
> The problem, in terms of it's behavior, that I have is that I want it to
> fail open (accept mail for a domain) when the smtp server that I'm
> checking against for a valid recipient is not responding at all.
> Unfortunately, this is not the case.  What I'm seeing is the email
> request is denied with a 450 at the backup mx server when the primary mx
> server is not responding.

Is there a good reason a 450 tempfail isn't good enough? The mail will
get retried later, and unless the target server is down for an extended
time, will be delivered eventually (usually not a huge delay).

--

Best regards,

Charles
Reply | Threaded
Open this post in threaded view
|

Re: Failing open with recipient address verification

Jay Deiman
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Charles Marcus wrote:
| On 6/23/2008, Jay Deiman ([hidden email]) wrote:
|> The problem, in terms of it's behavior, that I have is that I want it to
|> fail open (accept mail for a domain) when the smtp server that I'm
|> checking against for a valid recipient is not responding at all.
|> Unfortunately, this is not the case.  What I'm seeing is the email
|> request is denied with a 450 at the backup mx server when the primary mx
|> server is not responding.
|
| Is there a good reason a 450 tempfail isn't good enough? The mail will
| get retried later, and unless the target server is down for an extended
| time, will be delivered eventually (usually not a huge delay).

Yeah, I know what you mean there.  The problem is that the customers I'm
dealing with expect all their mail to be waiting on our servers (the
backup MXes) when they bring their mail server back up.  I'm kind of
stuck in this position as I, personally, don't really see a good reason
to have a backup MX in the first place short of an expected extended
downtime of the primary.

To be clear, personally I don't see any problem with the 450, but
according to requirements that I have for how things must work, I would
like to have this fail open on a failure to contact the primary MX.

Thanks for the response,

Jay Deiman


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIX9yEQ0lr+ZVKSBgRAiXzAJ4n/Gy0Hoh/6W6D0c3O+wW2EasVQQCcCBdr
ySsYfqwVgrikqXCaKzf/8XM=
=bhh4
-----END PGP SIGNATURE-----
Reply | Threaded
Open this post in threaded view
|

Re: Failing open with recipient address verification

Wietse Venema
In reply to this post by Charles Marcus
Charles Marcus:

> On 6/23/2008, Jay Deiman ([hidden email]) wrote:
> > The problem, in terms of it's behavior, that I have is that I want it to
> > fail open (accept mail for a domain) when the smtp server that I'm
> > checking against for a valid recipient is not responding at all.
> > Unfortunately, this is not the case.  What I'm seeing is the email
> > request is denied with a 450 at the backup mx server when the primary mx
> > server is not responding.
>
> Is there a good reason a 450 tempfail isn't good enough? The mail will
> get retried later, and unless the target server is down for an extended
> time, will be delivered eventually (usually not a huge delay).

Postfix 2.6 introduces the unverified_recipient_defer_code, which
allows you to override the default 450 reply.

According to documentation, smtp_fallback_relay can't be used here
because mail would almost certainly loop (the SMTP client's loop
detection does not distinguish between real mail and address probes).
For this reason, Postfix 2.2 and later ignores smtp_fallback_relay
for destinations that it is MX host for.

        Wietse

unverified_recipient_defer_code (default: 450)
       The numerical Postfix SMTP server response  when  a  recipient  address
       probe fails due to a temporary error condition.

       Unlike elsewhere in Postfix, you can specify 250 in order to accept the
       address anyway.

       Do not change this unless you have  a  complete  understanding  of  RFC
       2821.

       This feature is available in Postfix 2.6 and later.
Reply | Threaded
Open this post in threaded view
|

Re: Failing open with recipient address verification

mouss-2
In reply to this post by Jay Deiman
Jay Deiman wrote:

> Hello all,
>
> Here is a large problem I've been struggling with for a while now.  I'm
> managing some backup MX servers that, unfortunately, do not have lists
> of valid recipient addresses for the domains we are backing up.  Of
> course this means that they are a large source of backscatter, which is
> not at all what I want.
>
> Recently, I turned on recipient address verification and I have to say
> that I'm quite impressed with it.  It's just about exactly what I was
> looking for and solves all my issues, except for it's one downfall.
>
> The problem, in terms of it's behavior, that I have is that I want it to
> fail open (accept mail for a domain) when the smtp server that I'm
> checking against for a valid recipient is not responding at all.
> Unfortunately, this is not the case.  What I'm seeing is the email
> request is denied with a 450 at the backup mx server when the primary mx
> server is not responding.
>
> Is there any way around this that anyone knows of?  

if you really want to queue mail when remote server is down, then use a
monitoring daemon that check the remote server and updates the
configuration.

> I currently have
> tried using a different smtp transport in master.cf with a fallback
> relay set to a separate smtpd running on a different port on localhost
> and that does not seem to work as I had hoped.  I was hoping that it
> would fall back and verify at the local smtpd after the connection to
> the primary MX, which should respond OK.  Here is the relevant parts of
> my master.cf that I'm talking about:
>
> smtp      inet  n       -       n       -       500      smtpd
> 127.0.0.1:10025 inet n  -       n       -       500      smtpd
> ~    -o
> smtpd_recipient_restrictions=permit_mynetworks,reject_unauth_destination
>
> smtpver   unix  -       -       n       -       -       smtp
> ~    -o smtp_fallback_relay=127.0.0.1:10025
>
> In my address_verify_transport_maps:
>
> domain.org      smtpver:
>
> Thank you,
>
> Jay Deiman

Reply | Threaded
Open this post in threaded view
|

Re: Failing open with recipient address verification

Alex Bligh
In reply to this post by Jay Deiman


--On 23 June 2008 12:25:24 -0500 Jay Deiman <[hidden email]> wrote:

> To be clear, personally I don't see any problem with the 450, but
> according to requirements that I have for how things must work, I would
> like to have this fail open on a failure to contact the primary MX.

Perhaps a better heuristic (also not currently available in Postfix) would
be to accept mail to an address for which (a) an address verification
has already been carried out, and (b) the last such verification was
positive. IE 250 those which verified positively prior to the main MX
going down, 450 the others. This would obviously require Postfix maintaining
some form of database, but it would allow (1) most mail to be queued by
a backup MX and (2) not increase backscatter.

Alex
Reply | Threaded
Open this post in threaded view
|

Re: Failing open with recipient address verification

Victor Duchovni
On Mon, Jun 23, 2008 at 07:34:06PM +0100, Alex Bligh wrote:

>
>
> --On 23 June 2008 12:25:24 -0500 Jay Deiman <[hidden email]> wrote:
>
> >To be clear, personally I don't see any problem with the 450, but
> >according to requirements that I have for how things must work, I would
> >like to have this fail open on a failure to contact the primary MX.
>
> Perhaps a better heuristic (also not currently available in Postfix) would
> be to accept mail to an address for which (a) an address verification
> has already been carried out, and (b) the last such verification was
> positive. IE 250 those which verified positively prior to the main MX
> going down, 450 the others. This would obviously require Postfix maintaining
> some form of database, but it would allow (1) most mail to be queued by
> a backup MX and (2) not increase backscatter.
>

You should probably have read the documentation before posting this.

--
        Viktor.

Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.

To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:[hidden email]?body=unsubscribe%20postfix-users>

If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.
Reply | Threaded
Open this post in threaded view
|

Re: Failing open with recipient address verification

Jay Deiman
In reply to this post by Wietse Venema
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Wietse Venema wrote:
| Charles Marcus:
|> On 6/23/2008, Jay Deiman ([hidden email]) wrote:
|>> The problem, in terms of it's behavior, that I have is that I want it to
|>> fail open (accept mail for a domain) when the smtp server that I'm
|>> checking against for a valid recipient is not responding at all.
|>> Unfortunately, this is not the case.  What I'm seeing is the email
|>> request is denied with a 450 at the backup mx server when the primary mx
|>> server is not responding.
|> Is there a good reason a 450 tempfail isn't good enough? The mail will
|> get retried later, and unless the target server is down for an extended
|> time, will be delivered eventually (usually not a huge delay).
|
| Postfix 2.6 introduces the unverified_recipient_defer_code, which
| allows you to override the default 450 reply.
|
| According to documentation, smtp_fallback_relay can't be used here
| because mail would almost certainly loop (the SMTP client's loop
| detection does not distinguish between real mail and address probes).
| For this reason, Postfix 2.2 and later ignores smtp_fallback_relay
| for destinations that it is MX host for.
|
| Wietse
|
| unverified_recipient_defer_code (default: 450)
|        The numerical Postfix SMTP server response  when  a  recipient
  address
|        probe fails due to a temporary error condition.
|
|        Unlike elsewhere in Postfix, you can specify 250 in order to
accept the
|        address anyway.
|
|        Do not change this unless you have  a  complete  understanding
  of  RFC
|        2821.
|
|        This feature is available in Postfix 2.6 and later.

Yes, that is exactly what I was looking for.  I just tested it out and
it behaves perfectly.  Now I just have to wait for 2.6 to be marked
stable ;-)

Thanks for help Wietse,

Jay Deiman
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIX/R7Q0lr+ZVKSBgRAlZAAJ9XNZbjK6/I1lDZcwDjGmBWiSIBaACgkdrK
DzADg8CIf0RVp2xadtmFGQ0=
=dAz4
-----END PGP SIGNATURE-----
Reply | Threaded
Open this post in threaded view
|

Re: Failing open with recipient address verification

Wietse Venema
In reply to this post by Alex Bligh
Alex Bligh:

>
>
> --On 23 June 2008 12:25:24 -0500 Jay Deiman <[hidden email]> wrote:
>
> > To be clear, personally I don't see any problem with the 450, but
> > according to requirements that I have for how things must work, I would
> > like to have this fail open on a failure to contact the primary MX.
>
> Perhaps a better heuristic (also not currently available in Postfix) would
> be to accept mail to an address for which (a) an address verification
> has already been carried out, and (b) the last such verification was
> positive. IE 250 those which verified positively prior to the main MX
> going down,

That's what Postfix does already (but having a persistent address
verification database helps). Once an address "verifies" it is not
thrown out of the cache until it expires. Postfix will try to
re-verify an address before it expires, but it will ignore failed
re-verify probes.

> 450 the others. This would obviously require Postfix maintaining
> some form of database, but it would allow (1) most mail to be queued by
> a backup MX and (2) not increase backscatter.

The persistent address_verify_map feature. The only glitch is that
it has no background cleanup thread, so you would need to rotate
the file periodically and do "postfix reload".

        Wietse

address_verify_map (default: empty)
       Optional lookup table for persistent address verification status  stor-
       age.   The  table is maintained by the verify(8) service, and is opened
       before the process releases privileges.

       By default, the information is kept in volatile  memory,  and  is  lost
       after "postfix reload" or "postfix stop".

       Specify a location in a file system that will not fill up. If the data-
       base becomes corrupted, the world comes to an end.  To  recover  delete
       the file and do "postfix reload".

       As  of version 2.5, Postfix no longer uses root privileges when opening
       this file. The file  should  now  be  stored  under  the  Postfix-owned
       data_directory. As a migration aid, an attempt to open the file under a
       non-Postfix directory is redirected to  the  Postfix-owned  data_direc-
       tory, and a warning is logged.

       Examples:

       address_verify_map = btree:/var/lib/postfix/verify

Reply | Threaded
Open this post in threaded view
|

Re: Failing open with recipient address verification

Alex Bligh


--On 23 June 2008 15:51:02 -0400 Wietse Venema <[hidden email]> wrote:

> That's what Postfix does already

Serves me right for not reading the manual.

Alex