gmail servers requiring postscreen_access whitelisting

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

gmail servers requiring postscreen_access whitelisting

Curtis Villamizar
Since I enabled postscreen (with soft_bounce=yes in master.cf) I was
getting logs of this form:

Apr  9 01:08:12 mta1 postfix/postscreen[18326]:
  NOQUEUE: reject: RCPT from [2607:f8b0:4002:c05::22d]:32999:
  450 4.3.2 Service currently unavailable;
  from=<[hidden email]>, to=<REDACTED>,
  proto=ESMTP, helo=<mail-yw0-x22d.google.com>

linefeeds added by me for readability.

gmail would just keep trying a half hour later and mail never gets
delivered.

rfc3463 isn't very helpful:

  X.3.2   System not accepting network messages

    The host on which the mailbox is resident is not accepting
    messages.  Examples of such conditions include an immanent
    shutdown, excessive load, or system maintenance.  This is
    useful for both permanent and permanent transient errors.

I have lines of the form:

  main.cf:
  postscreen_access_list =
      cidr:$config_directory/postscreen_access
      hash:$config_directory/postscreen_reject

  postscreen_access:
  #  google mail servers
  2607:f8b0:4002:c00::/60         permit
  [... other google server blocks ...]

This is a workaround that shouldn't be needed.

Any idea what the cause of this is?  So far no legit mail except gmail
gets caught here.

Curtis
Reply | Threaded
Open this post in threaded view
|

Re: gmail servers requiring postscreen_access whitelisting

Noel Jones-2
On 4/9/2016 10:00 PM, Curtis Villamizar wrote:

> Since I enabled postscreen (with soft_bounce=yes in master.cf) I was
> getting logs of this form:
>
> Apr  9 01:08:12 mta1 postfix/postscreen[18326]:
>   NOQUEUE: reject: RCPT from [2607:f8b0:4002:c05::22d]:32999:
>   450 4.3.2 Service currently unavailable;
>   from=<[hidden email]>, to=<REDACTED>,
>   proto=ESMTP, helo=<mail-yw0-x22d.google.com>
>
> linefeeds added by me for readability.
>
> gmail would just keep trying a half hour later and mail never gets
> delivered.
>
> rfc3463 isn't very helpful:
>
>   X.3.2   System not accepting network messages
>
>     The host on which the mailbox is resident is not accepting
>     messages.  Examples of such conditions include an immanent
>     shutdown, excessive load, or system maintenance.  This is
>     useful for both permanent and permanent transient errors.
>
> I have lines of the form:
>
>   main.cf:
>   postscreen_access_list =
>       cidr:$config_directory/postscreen_access
>       hash:$config_directory/postscreen_reject
>
>   postscreen_access:
>   #  google mail servers
>   2607:f8b0:4002:c00::/60         permit
>   [... other google server blocks ...]
>
> This is a workaround that shouldn't be needed.
>
> Any idea what the cause of this is?  So far no legit mail except gmail
> gets caught here.
>
> Curtis
>


Look for other warnings and errors in your logs, maybe just before
the reject, maybe earlier.



  -- Noel Jones
Reply | Threaded
Open this post in threaded view
|

Re: gmail servers requiring postscreen_access whitelisting

Curtis Villamizar
In message <[hidden email]>
Noel Jones writes:
 

> On 4/9/2016 10:00 PM, Curtis Villamizar wrote:
> > Since I enabled postscreen (with soft_bounce=yes in master.cf) I was
> > getting logs of this form:
> >
> > Apr  9 01:08:12 mta1 postfix/postscreen[18326]:
> >   NOQUEUE: reject: RCPT from [2607:f8b0:4002:c05::22d]:32999:
> >   450 4.3.2 Service currently unavailable;
> >   from=<[hidden email]>, to=<REDACTED>,
> >   proto=ESMTP, helo=<mail-yw0-x22d.google.com>
> >
> > linefeeds added by me for readability.
> >
> > gmail would just keep trying a half hour later and mail never gets
> > delivered.
> >
> > rfc3463 isn't very helpful:
> >
> >   X.3.2   System not accepting network messages
> >
> >     The host on which the mailbox is resident is not accepting
> >     messages.  Examples of such conditions include an immanent
> >     shutdown, excessive load, or system maintenance.  This is
> >     useful for both permanent and permanent transient errors.
> >
> > I have lines of the form:
> >
> >   main.cf:
> >   postscreen_access_list =
> >       cidr:$config_directory/postscreen_access
> >       hash:$config_directory/postscreen_reject
> >
> >   postscreen_access:
> >   #  google mail servers
> >   2607:f8b0:4002:c00::/60         permit
> >   [... other google server blocks ...]
> >
> > This is a workaround that shouldn't be needed.
> >
> > Any idea what the cause of this is?  So far no legit mail except gmail
> > gets caught here.
> >
> > Curtis
>  
> Look for other warnings and errors in your logs, maybe just before
> the reject, maybe earlier.
>  
>   -- Noel Jones


This is it for that connections:

Apr  9 01:07:15 mta1 postfix/postscreen[18326]: CONNECT from
  [2607:f8b0:4002:c05::22d]:32999 to [2001:470:88e6:1::141]:25
Apr  9 01:07:18 mta1 postfix/tlsproxy[18331]: CONNECT from
  [2607:f8b0:4002:c05::22d]:32999
Apr  9 01:08:12 mta1 postfix/tlsproxy[18331]: Untrusted TLS connection
  established from [2607:f8b0:4002:c05::22d]:32999: TLSv1.2 with cipher
  ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Apr  9 01:08:12 mta1 postfix/postscreen[18326]: NOQUEUE: reject: RCPT
  from [2607:f8b0:4002:c05::22d]:32999: 450 4.3.2 Service currently
  unavailable; from=<[hidden email]>, to=<REDACTED>,
  proto=ESMTP, helo=<mail-yw0-x22d.google.com>
Apr  9 01:08:12 mta1 postfix/postscreen[18326]: PASS NEW
  [2607:f8b0:4002:c05::22d]:32999
Apr  9 01:08:12 mta1 postfix/tlsproxy[18331]: DISCONNECT
  [2607:f8b0:4002:c05::22d]:32999
Apr  9 01:08:12 mta1 postfix/postscreen[18326]: DISCONNECT
  [2607:f8b0:4002:c05::22d]:32999

The IP address got whitelisted but then the next retry from gmail
usually doesn't come from the same IP address and comes 30 minutes
later.  They seem to have some sort of pool of servers that work on
the same set of mail queues.  Today I caught 2 gmails where this
happenned where I didn't have the block in the permit list but each
got delivered on next attempt.

I haven't had postscreen enabled long and only for two domain, one
currently used only for a web site and therefore available for email
testing and the other that is mostly mail to me and gets a fair amount
of spam.

I now have a non-gmail sender where this happened.  In that case after
the 450 it went immediately to the secondary MX that at this time is
not running postscreen and all was fine.

I'll recheck my configs, then post if I can't figure it out.

Curtis
Reply | Threaded
Open this post in threaded view
|

Re: gmail servers requiring postscreen_access whitelisting

Peter Ajamian
In reply to this post by Curtis Villamizar
On 10/04/16 15:00, Curtis Villamizar wrote:
> This is a workaround that shouldn't be needed.
>
> Any idea what the cause of this is?  So far no legit mail except gmail
> gets caught here.

gmail uses hundreds, or thousands of MTAs and has the unique property
that when they retry after a deferral it is almost always from a
different server (IP).  So postfix clears one IP and they retry from
another which postfix did not clear yet.  Rinse and repeat ad-nauseum.

The only workaround is to either receive so much mail from google that
you eventually get most of their servers on your temporary whitelist, or
to whitelist them in some other way.  newer versions of postfix allow
you to whitelist based on DNSWLS and if you use dnswl.org it will
include the google servers.  In older versions of postfix you will need
to whitelist them manually like you have already done, but they change
from time to time so you need to keep the list up to date.


Peter
Reply | Threaded
Open this post in threaded view
|

Re: gmail servers requiring postscreen_access whitelisting

Bastian Blank-3
In reply to this post by Curtis Villamizar
On Sat, Apr 09, 2016 at 11:00:53PM -0400, Curtis Villamizar wrote:
> Any idea what the cause of this is?  So far no legit mail except gmail
> gets caught here.

Don't use after-greeting tests in postscreen.  The postscreen
documentation explains exactly why this happens.

Bastian

--
        "What terrible way to die."
        "There are no good ways."
                -- Sulu and Kirk, "That Which Survives", stardate unknown
Reply | Threaded
Open this post in threaded view
|

Re: gmail servers requiring postscreen_access whitelisting

Wietse Venema
In reply to this post by Curtis Villamizar
Curtis Villamizar:
> Since I enabled postscreen (with soft_bounce=yes in master.cf) I was
> getting logs of this form:
>
> Apr  9 01:08:12 mta1 postfix/postscreen[18326]:
>   NOQUEUE: reject: RCPT from [2607:f8b0:4002:c05::22d]:32999:
>   450 4.3.2 Service currently unavailable;
>   from=<[hidden email]>, to=<REDACTED>,
>   proto=ESMTP, helo=<mail-yw0-x22d.google.com>

DO NOT turn on the after-220 tests:

postscreen_non_smtp_command_enable = no
postscreen_pipelining_enable = no
postscreen_bare_newline_enable = no

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: gmail servers requiring postscreen_access whitelisting

Curtis Villamizar
In message <[hidden email]>
Wietse Venema writes:

>
> Curtis Villamizar:
> > Since I enabled postscreen (with soft_bounce=yes in master.cf) I was
> > getting logs of this form:
> >
> > Apr  9 01:08:12 mta1 postfix/postscreen[18326]:
> >   NOQUEUE: reject: RCPT from [2607:f8b0:4002:c05::22d]:32999:
> >   450 4.3.2 Service currently unavailable;
> >   from=<[hidden email]>, to=<REDACTED>,
> >   proto=ESMTP, helo=<mail-yw0-x22d.google.com>
>  
> DO NOT turn on the after-220 tests:
>  
> postscreen_non_smtp_command_enable = no
> postscreen_pipelining_enable = no
> postscreen_bare_newline_enable = no
>  
> Wietse


Based on the terseness of this authoritative response I'll assume this
topic has already been beat to death and the limited benefits of these
tests simply are not worth the headaches.

Peter gave a useful workaround.  I'll reply on that thread.

I had enabled _pipelining_ and _non_smtp_command_ but not
_bare_newline_.  For now I've disabled these tests.

Curtis
Reply | Threaded
Open this post in threaded view
|

Re: gmail servers requiring postscreen_access whitelisting

Curtis Villamizar
In reply to this post by Peter Ajamian
In message <[hidden email]>
Peter writes:

>
> On 10/04/16 15:00, Curtis Villamizar wrote:
> > This is a workaround that shouldn't be needed.
> >
> > Any idea what the cause of this is?  So far no legit mail except gmail
> > gets caught here.
>  
> gmail uses hundreds, or thousands of MTAs and has the unique property
> that when they retry after a deferral it is almost always from a
> different server (IP).  So postfix clears one IP and they retry from
> another which postfix did not clear yet.  Rinse and repeat ad-nauseum.
>  
> The only workaround is to either receive so much mail from google that
> you eventually get most of their servers on your temporary whitelist, or
> to whitelist them in some other way.  newer versions of postfix allow
> you to whitelist based on DNSWLS and if you use dnswl.org it will
> include the google servers.  In older versions of postfix you will need
> to whitelist them manually like you have already done, but they change
> from time to time so you need to keep the list up to date.
>  
> Peter


This seems like it could be a viable workaround for the after 220
problem.

  postscreen_dnsbl_sites =
      list.dnswl.org*-5
      # followed by some blacklist sites

It could occasionally delay mail from all legitimate senders not in
dnswl.org (almost everyone but a few big guys) if they try both the
primary and secondary MX and those two MX have independent temporary
whitelists.  Tying the temporary whitelists together (so the secondary
immediately passes postscreen tests) using a routable address (since
they are at different sites) seems horribly insecure.  If there was a
way to wrap the connection in TLS, then OK.

Occasionally delaying legitimate mail is to be avoided and I don't see
a workaround for that.

OTOH, as soon as I turned this off some obvious spam got through,
probably bot spam not yet listed in a dnsbl and clever enough to not
get snagged by spamassasin (not all that hard apparently).

The next question is whether the after 220 stops enough spam that the
other tests wouldn't get to make it worth the bother.  Apparently,
based on Wietse's terse comment, he thinks not.

So I'll go with Wietse's advice and disable after 220 tests and see if
I can find an alternative to stop the remaining dribble of spam.

Curtis
Reply | Threaded
Open this post in threaded view
|

Re: gmail servers requiring postscreen_access whitelisting

@lbutlr
On Apr 10, 2016, at 10:24 AM, Curtis Villamizar <[hidden email]> wrote:
>  postscreen_dnsbl_sites =
>      list.dnswl.org*-5
>      # followed by some blacklist sites

It was my understanding that eh the order of test said not matter because all the dnsbls listed would be checked, a final score computed, and then that compound number passed along to postscreen.

--
Rid yourself of doubt -- or should you? -George Carlin

Reply | Threaded
Open this post in threaded view
|

Re: gmail servers requiring postscreen_access whitelisting

Wietse Venema
@lbutlr:
> On Apr 10, 2016, at 10:24 AM, Curtis Villamizar =
> <[hidden email]> wrote:
> >  postscreen_dnsbl_sites =3D
> >      list.dnswl.org*-5
> >      # followed by some blacklist sites
>
> It was my understanding that eh the order of test said not matter =
> because all the dnsbls listed would be checked, a final score computed, =
> and then that compound number passed along to postscreen.

Correct. The lookups are done in parallel, and the score is updated
as the results become available. As long as the numbers are small
enough (no integer over/underflow), the update order does not matter.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: gmail servers requiring postscreen_access whitelisting

Curtis Villamizar
In reply to this post by @lbutlr
In message <[hidden email]>
"@lbutlr" writes:

>
> On Apr 10, 2016, at 10:24 AM, Curtis Villamizar =
> <[hidden email]> wrote:
> >  postscreen_dnsbl_sites =3D
> >      list.dnswl.org*-5
> >      # followed by some blacklist sites
>  
> It was my understanding that eh the order of test said not matter
> because all the dnsbls listed would be checked, a final score
> computed, and then that compound number passed along to postscreen.

Nobody ever said there was an order dependence.

btw- I don't think list.dnswl.org is a viable workaround for the post
220 problem.  This just affects the dnsbl score which would already be
zero.  The post 220 checks would still be run before putting the gmail
server IP into the temporary whitelist.  Manual maintenance of
postscreen_access is the only thing that would work.

Curtis

Reply | Threaded
Open this post in threaded view
|

Re: gmail servers requiring postscreen_access whitelisting

Peter Ajamian
On 11/04/16 11:37, Curtis Villamizar wrote:
> btw- I don't think list.dnswl.org is a viable workaround for the post
> 220 problem.  This just affects the dnsbl score which would already be
> zero.  The post 220 checks would still be run before putting the gmail
> server IP into the temporary whitelist.  Manual maintenance of
> postscreen_access is the only thing that would work.

You need postfix 2.11 and later then you can use
postscreen_dnsbl_whitelist_threshold to allow whitelisted servers skip
the after-220 tests and therefore not get delayed.


Peter
Reply | Threaded
Open this post in threaded view
|

Re: gmail servers requiring postscreen_access whitelisting

Wietse Venema
In reply to this post by Curtis Villamizar
Curtis Villamizar:
> btw- I don't think list.dnswl.org is a viable workaround for the post
> 220 problem.  This just affects the dnsbl score which would already be
> zero.  The post 220 checks would still be run before putting the gmail
> server IP into the temporary whitelist.  Manual maintenance of
> postscreen_access is the only thing that would work.

With postscreen_dnsbl_whitelist_threshold you can pre-empt subsequent checks.

postscreen_dnsbl_whitelist_threshold (default: 0)
   Allow a remote SMTP client to skip "before" and  "after  220  greeting"
   protocol  tests,  based on its combined DNSBL score as defined with the
   postscreen_dnsbl_sites parameter.

   Specify a negative value to enable this feature. When a  client  passes
   the  postscreen_dnsbl_whitelist_threshold  without  having failed other
   tests, all pending or disabled tests are flagged as  completed  with  a
   time-to-live  value  equal  to  postscreen_dnsbl_ttl.   When a test was
   already completed, its time-to-live value is updated  if  it  was  less
   than postscreen_dnsbl_ttl.

   This feature is available in Postfix 2.11.

It was desigined precisely for this purpose.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: gmail servers requiring postscreen_access whitelisting

@lbutlr
In reply to this post by Curtis Villamizar

> On Apr 10, 2016, at 5:37 PM, Curtis Villamizar <[hidden email]> wrote:
>
> In message <[hidden email]>
> "@lbutlr" writes:
>>
>> On Apr 10, 2016, at 10:24 AM, Curtis Villamizar =
>> <[hidden email]> wrote:
>>> postscreen_dnsbl_sites =3D
>>>     list.dnswl.org*-5
>>>     # followed by some blacklist sites
>>
>> It was my understanding that eh the order of test said not matter
>> because all the dnsbls listed would be checked, a final score
>> computed, and then that compound number passed along to postscreen.
>
> Nobody ever said there was an order dependence.

“Followed by” does imply that order may be significant.

>
> btw- I don't think list.dnswl.org is a viable workaround for the post
> 220 problem.  This just affects the dnsbl score which would already be
> zero.  The post 220 checks would still be run before putting the gmail
> server IP into the temporary whitelist.  Manual maintenance of
> postscreen_access is the only thing that would work.

Isn’t it that if an IP hasn’t been seen and scores 0 postscreeen sends a temporary failure, so scoring it negative means it gets an immediate pass?

I know that enabling post screen and dnswl stopped the issues with large mailers on my system.


--
Nothing gold can stay -- Robert Frost Stay gold -- Johnny Cade

Reply | Threaded
Open this post in threaded view
|

Re: gmail servers requiring postscreen_access whitelisting

Steve Jenkins-2
On Tue, Apr 12, 2016 at 4:30 AM, @lbutlr <[hidden email]> wrote:

> On Apr 10, 2016, at 5:37 PM, Curtis Villamizar <[hidden email]> wrote:
>
> In message <[hidden email]>
> "@lbutlr" writes:
>>
>> On Apr 10, 2016, at 10:24 AM, Curtis Villamizar =
>> <[hidden email]> wrote:
>>> postscreen_dnsbl_sites =3D
>>>     list.dnswl.org*-5
>>>     # followed by some blacklist sites
>>
>> It was my understanding that eh the order of test said not matter
>> because all the dnsbls listed would be checked, a final score
>> computed, and then that compound number passed along to postscreen.
>
> Nobody ever said there was an order dependence.

“Followed by” does imply that order may be significant.

>
> btw- I don't think list.dnswl.org is a viable workaround for the post
> 220 problem.  This just affects the dnsbl score which would already be
> zero.  The post 220 checks would still be run before putting the gmail
> server IP into the temporary whitelist.  Manual maintenance of
> postscreen_access is the only thing that would work.

Isn’t it that if an IP hasn’t been seen and scores 0 postscreeen sends a temporary failure, so scoring it negative means it gets an immediate pass?

I know that enabling post screen and dnswl stopped the issues with large mailers on my system.

Curtis:

+1 to the suggestion of properly using dnswl.org. But if you'd also like to automatically scan the SPF records of mailers you trust (including Gmail) and build an up-to-date Postscreen whitelist based on their published SMTP servers, then Postwhite maybe of interest to you:


SteveJ