Quantcast

Optimising new system and postscreen questions

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Optimising new system and postscreen questions

Simon Wilson-7
On my new Postfix 2.10 system incoming mail is slow to process (about  
15 seconds end to end), and I think it is mainly because DNS queries  
are slowing things down.

The server runs local caching DNS BIND, so it's as quick as I can get  
it on the slow Internet connection we are on.

At the moment it seems like every step along the inbound email process  
is doing separate DNSBL lookups, and I'm wondering if this can be  
streamlined.

Postscreen runs first and takes pretty much 6 seconds every time:

May  1 18:19:20 emp07 postfix/postscreen[28420]: CONNECT from  
[64.20.227.131]:57512 to [192.168.1.235]:25
May  1 18:19:26 emp07 postfix/postscreen[28420]: PASS NEW  
[64.20.227.131]:57512

Postscreen is using (threshold 3):

         zen.spamhaus.org*3
         bl.mailspike.net*2
         b.barracudacentral.org*2
         bl.spameatingmonkey.net
         bl.spamcop.net
         dnsbl.sorbs.net
         hostkarma.junkemailfilter.com=127.0.0.2
         hostkarma.junkemailfilter.com=127.0.0.4
         hostkarma.junkemailfilter.com=127.0.1.2
         psbl.surriel.com
         swl.spamhaus.org*-4
         list.dnswl.org=127.0.[2..15].0*-2
         list.dnswl.org=127.0.[2..15].1*-3
         list.dnswl.org=127.0.[2..15].[2..3]*-4
         wl.mailspike.net=127.0.0.[17;18]*-1
         wl.mailspike.net=127.0.0.[19;20]*-2
         hostkarma.junkemailfilter.com=127.0.0.1*-1

(Yes I've checked them all, including 'registering' where necessary  
for some of those).

Some of them register response in the logs - are the rest timing out?

May  1 18:38:30 emp07 postfix/postscreen[29413]: CONNECT from  
[64.20.227.134]:60378 to [192.168.1.235]:25
May  1 18:38:30 emp07 postfix/dnsblog[29423]: addr 64.20.227.134  
listed by domain hostkarma.junkemailfilter.com as 127.0.0.1
May  1 18:38:30 emp07 postfix/dnsblog[29423]: addr 64.20.227.134  
listed by domain hostkarma.junkemailfilter.com as 127.0.1.1
May  1 18:38:30 emp07 postfix/dnsblog[29418]: addr 64.20.227.134  
listed by domain list.dnswl.org as 127.0.3.1
May  1 18:38:31 emp07 postfix/dnsblog[29421]: addr 64.20.227.134  
listed by domain dnsbl.sorbs.net as 127.0.0.7
May  1 18:38:36 emp07 postfix/postscreen[29413]: PASS NEW  
[64.20.227.134]:60378


Then Postfix smtpd takes 3 to 4 seconds to get to 'cleanup' stage,  
including an SPF-policy lookup and a reject_rbl_client  
zen.spamhaus.org line.

Then amavisd-new runs, and spamassassin does more BL lookups,  
including on URIs in the email (3 or 4 seconds there too).

End result is 15 seconds or so end to end before it gets delivered.

Most of the time this is fine, the server is low volume. However it  
got me thinking about all the separate DNS lookups...

1. Would postscreen lose much effectiveness by taking some of the lookups out?
2. Is the reject_rbl_client zen.spamhaus.org doing anything when  
postscreen has already done a zen.spamhaus lookup?
3. Any other ways to speed it up, or should I accept the trade-off  
between speed and accuracy of result?
4. Is it worth running postscreen in more detailed (verbose?) mode to  
see what it is doing?

Simon.



--
Simon Wilson
M: 0400 12 11 16

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Optimising new system and postscreen questions

Simon Wilson-7
----- Message from Simon Wilson <[hidden email]> ---------
     Date: Mon, 01 May 2017 18:43:41 +1000
     From: Simon Wilson <[hidden email]>
Reply-To: [hidden email]
  Subject: Optimising new system and postscreen questions
       To: Postfix users <[hidden email]>


> On my new Postfix 2.10 system incoming mail is slow to process  
> (about 15 seconds end to end), and I think it is mainly because DNS  
> queries are slowing things down.
>
> The server runs local caching DNS BIND, so it's as quick as I can  
> get it on the slow Internet connection we are on.
>
> At the moment it seems like every step along the inbound email  
> process is doing separate DNSBL lookups, and I'm wondering if this  
> can be streamlined.
>
> Postscreen runs first and takes pretty much 6 seconds every time:
>
> May  1 18:19:20 emp07 postfix/postscreen[28420]: CONNECT from  
> [64.20.227.131]:57512 to [192.168.1.235]:25
> May  1 18:19:26 emp07 postfix/postscreen[28420]: PASS NEW  
> [64.20.227.131]:57512
>
> Postscreen is using (threshold 3):
>
>         zen.spamhaus.org*3
>         bl.mailspike.net*2
>         b.barracudacentral.org*2
>         bl.spameatingmonkey.net
>         bl.spamcop.net
>         dnsbl.sorbs.net
>         hostkarma.junkemailfilter.com=127.0.0.2
>         hostkarma.junkemailfilter.com=127.0.0.4
>         hostkarma.junkemailfilter.com=127.0.1.2
>         psbl.surriel.com
>         swl.spamhaus.org*-4
>         list.dnswl.org=127.0.[2..15].0*-2
>         list.dnswl.org=127.0.[2..15].1*-3
>         list.dnswl.org=127.0.[2..15].[2..3]*-4
>         wl.mailspike.net=127.0.0.[17;18]*-1
>         wl.mailspike.net=127.0.0.[19;20]*-2
>         hostkarma.junkemailfilter.com=127.0.0.1*-1
>
> (Yes I've checked them all, including 'registering' where necessary  
> for some of those).
>
> Some of them register response in the logs - are the rest timing out?
>
> May  1 18:38:30 emp07 postfix/postscreen[29413]: CONNECT from  
> [64.20.227.134]:60378 to [192.168.1.235]:25
> May  1 18:38:30 emp07 postfix/dnsblog[29423]: addr 64.20.227.134  
> listed by domain hostkarma.junkemailfilter.com as 127.0.0.1
> May  1 18:38:30 emp07 postfix/dnsblog[29423]: addr 64.20.227.134  
> listed by domain hostkarma.junkemailfilter.com as 127.0.1.1
> May  1 18:38:30 emp07 postfix/dnsblog[29418]: addr 64.20.227.134  
> listed by domain list.dnswl.org as 127.0.3.1
> May  1 18:38:31 emp07 postfix/dnsblog[29421]: addr 64.20.227.134  
> listed by domain dnsbl.sorbs.net as 127.0.0.7
> May  1 18:38:36 emp07 postfix/postscreen[29413]: PASS NEW  
> [64.20.227.134]:60378
>
>
> Then Postfix smtpd takes 3 to 4 seconds to get to 'cleanup' stage,  
> including an SPF-policy lookup and a reject_rbl_client  
> zen.spamhaus.org line.
>
> Then amavisd-new runs, and spamassassin does more BL lookups,  
> including on URIs in the email (3 or 4 seconds there too).
>
> End result is 15 seconds or so end to end before it gets delivered.
>
> Most of the time this is fine, the server is low volume. However it  
> got me thinking about all the separate DNS lookups...
>
> 1. Would postscreen lose much effectiveness by taking some of the  
> lookups out?
> 2. Is the reject_rbl_client zen.spamhaus.org doing anything when  
> postscreen has already done a zen.spamhaus lookup?
> 3. Any other ways to speed it up, or should I accept the trade-off  
> between speed and accuracy of result?
> 4. Is it worth running postscreen in more detailed (verbose?) mode  
> to see what it is doing?
>
> Simon.
>

I just realised postscreen_greet_wait (default: normal: 6s, overload:  
2s) will be the postscreen 6 seconds, as I have not over-ridden the  
default.

So that answers question 4... it's done the lookups, printed the  
results it got, and is now doing the postscren_greet_wait.

Simon

--
Simon Wilson
M: 0400 12 11 16

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Optimising new system and postscreen questions

Marco Pizzoli
In reply to this post by Simon Wilson-7
Hello Simon,

The server runs local caching DNS BIND, so it's as quick as I can get it on the slow Internet connection we are on.

I don't qualify mysef expert enough to answer the rest of your points, but for the DNS part I suggest you think about replacing BIND with Unbound, as the DNS resolver. It has a property called min_ttl that permits you to impose a minimum amount of TTL to the entries reported. DNSBL have always real low TTL values, on purpose. If you are fne with relaxing this real-timeness, well by setting a value of i.e. 60/90 seconds it will permit you to reduce the network dependency.

Worth a try.
Marco
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Optimising new system and postscreen questions

Simon Wilson-7
----- Message from Marco Pizzoli <[hidden email]> ---------
    Date: Mon, 1 May 2017 11:18:30 +0200
    From: Marco Pizzoli <[hidden email]>
Subject: Re: Optimising new system and postscreen questions
      To: [hidden email]
      Cc: Postfix users <[hidden email]>


> Hello Simon,
>
> The server runs local caching DNS BIND, so it's as quick as I can get it on
>> the slow Internet connection we are on.
>>
>
> I don't qualify mysef expert enough to answer the rest of your points, but
> for the DNS part I suggest you think about replacing BIND with Unbound, as
> the DNS resolver. It has a property called min_ttl that permits you to
> impose a minimum amount of TTL to the entries reported. DNSBL have always
> real low TTL values, on purpose. If you are fne with relaxing this
> real-timeness, well by setting a value of i.e. 60/90 seconds it will permit
> you to reduce the network dependency.
>
> Worth a try.
> Marco

Thanks Marco, I'll investigate that.  :)

Simon

--
Simon Wilson
M: 0400 12 11 16

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Optimising new system and postscreen questions

L.P.H. van Belle

And if you running debian you can set the min-cache-ttl..

That bind is patched with :
https://anonscm.debian.org/cgit/users/lamont/bind9.git/commit/?h=patches&id=84fa402750fab5cd887d357501e2896494ac551f


So you can set these if needed.
min-cache-ttl 90;
min-ncache-ttl 90;


Greetz,

Louis


> -----Oorspronkelijk bericht-----
> Van: [hidden email]
> [mailto:[hidden email]] Namens Simon Wilson
> Verzonden: maandag 1 mei 2017 11:20
> Aan: Marco Pizzoli
> CC: Postfix users
> Onderwerp: Re: Optimising new system and postscreen questions
>
> ----- Message from Marco Pizzoli <[hidden email]> ---------
>     Date: Mon, 1 May 2017 11:18:30 +0200
>     From: Marco Pizzoli <[hidden email]>
> Subject: Re: Optimising new system and postscreen questions
>       To: [hidden email]
>       Cc: Postfix users <[hidden email]>
>
>
> > Hello Simon,
> >
> > The server runs local caching DNS BIND, so it's as quick as
> I can get
> > it on
> >> the slow Internet connection we are on.
> >>
> >
> > I don't qualify mysef expert enough to answer the rest of
> your points,
> > but for the DNS part I suggest you think about replacing BIND with
> > Unbound, as the DNS resolver. It has a property called min_ttl that
> > permits you to impose a minimum amount of TTL to the
> entries reported.
> > DNSBL have always real low TTL values, on purpose. If you
> are fne with
> > relaxing this real-timeness, well by setting a value of i.e. 60/90
> > seconds it will permit you to reduce the network dependency.
> >
> > Worth a try.
> > Marco
>
> Thanks Marco, I'll investigate that.  :)
>
> Simon
>
> --
> Simon Wilson
> M: 0400 12 11 16
>
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Optimising new system and postscreen questions

Wietse Venema
In reply to this post by Simon Wilson-7
Simon Wilson:

> On my new Postfix 2.10 system incoming mail is slow to process (about  
> 15 seconds end to end), and I think it is mainly because DNS queries  
> are slowing things down.
>
> The server runs local caching DNS BIND, so it's as quick as I can get  
> it on the slow Internet connection we are on.
>
> At the moment it seems like every step along the inbound email process  
> is doing separate DNSBL lookups, and I'm wondering if this can be  
> streamlined.
>
> Postscreen runs first and takes pretty much 6 seconds every time:

That is incorrect. The 6 seconds is only for clients that haven't
passed the pregreet test in the past 24 hours (the time is controlled
with the postscreen_greet_ttl parameter).

        Wietse
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Optimising new system and postscreen questions

Simon Wilson-7
> Simon Wilson:
>> On my new Postfix 2.10 system incoming mail is slow to process (about
>> 15 seconds end to end), and I think it is mainly because DNS queries
>> are slowing things down.
>>
>> The server runs local caching DNS BIND, so it's as quick as I can get
>> it on the slow Internet connection we are on.
>>
>> At the moment it seems like every step along the inbound email process
>> is doing separate DNSBL lookups, and I'm wondering if this can be
>> streamlined.
>>
>> Postscreen runs first and takes pretty much 6 seconds every time:
>
> That is incorrect. The 6 seconds is only for clients that haven't
> passed the pregreet test in the past 24 hours (the time is controlled
> with the postscreen_greet_ttl parameter).
>
> Wietse

THanks Wietse - yes I noticed that after I sent the first email to  
list, and corrected myself with a follow-up.

I have to say that it is *very* cool seeing postscreen do its stuff...  
I've been using postfix for many years with postgrey, so this is a new  
direction for me.

Still interested to see if anyone can comment on my other questions though:

1. Would postscreen lose much effectiveness by taking some of the  
lookups out? - I think I can answer that one myself now that I know  
how fast postscreen is actually running, but if anyone can see any  
obvious problems with my combination of BLs please let me know (listed  
again at the end). I can't use the whitelist threshold (only on 2.10).

2. Is the reject_rbl_client zen.spamhaus.org doing anything when  
postscreen has already done a zen.spamhaus lookup? This one I am keen  
to hear thoughts on. I'll keep an eye on logs and see if anything that  
gets through postscreen's zen lookup gets kicked by this one.

3. Any other ways to speed it up, or should I accept the trade-off  
between speed and accuracy of result?



Postscreen is using (threshold 3):

         zen.spamhaus.org*3
         bl.mailspike.net*2
         b.barracudacentral.org*2
         bl.spameatingmonkey.net
         bl.spamcop.net
         dnsbl.sorbs.net
         hostkarma.junkemailfilter.com=127.0.0.2
         hostkarma.junkemailfilter.com=127.0.0.4
         hostkarma.junkemailfilter.com=127.0.1.2
         psbl.surriel.com
         swl.spamhaus.org*-4
         list.dnswl.org=127.0.[2..15].0*-2
         list.dnswl.org=127.0.[2..15].1*-3
         list.dnswl.org=127.0.[2..15].[2..3]*-4
         wl.mailspike.net=127.0.0.[17;18]*-1
         wl.mailspike.net=127.0.0.[19;20]*-2
         hostkarma.junkemailfilter.com=127.0.0.1*-1

--
Simon Wilson
M: 0400 12 11 16

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Optimising new system and postscreen questions

Viktor Dukhovni

> On May 1, 2017, at 8:17 AM, Simon Wilson <[hidden email]> wrote:
>
> ostscreen is using (threshold 3):
>
>        zen.spamhaus.org*3
>        bl.mailspike.net*2
>        b.barracudacentral.org*2
>        bl.spameatingmonkey.net
>        bl.spamcop.net
>        dnsbl.sorbs.net
>        hostkarma.junkemailfilter.com=127.0.0.2
>        hostkarma.junkemailfilter.com=127.0.0.4
>        hostkarma.junkemailfilter.com=127.0.1.2
>        psbl.surriel.com
>        swl.spamhaus.org*-4
>        list.dnswl.org=127.0.[2..15].0*-2
>        list.dnswl.org=127.0.[2..15].1*-3
>        list.dnswl.org=127.0.[2..15].[2..3]*-4
>        wl.mailspike.net=127.0.0.[17;18]*-1
>        wl.mailspike.net=127.0.0.[19;20]*-2
>        hostkarma.junkemailfilter.com=127.0.0.1*-1

You'll likely find that after zen.spamhaus.org and bl.barracudacentral + bl.spamcop.net
you don't need any other RBLs, as they contribute almost nothing to the effectiveness
of the filter.  Throw in a single whitelist, and you're done.  I think that the current
list of RBLs is too large.  Start with a short list, grow with care one at a time if
needed, and only if effectiveness increases non-trivially without too many FPs.

As for a system that's too slow overall, have you checked whether your syslog service
might be a bottleneck?  Make sure that log writes are not synchronous.  With syslog-ng
use "unix-dgram" NOT "unix-stream".  I've no experience with systemd's logging, check
for troubles there if applicable.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Optimising new system and postscreen questions

Wietse Venema
Viktor Dukhovni:

>
> > On May 1, 2017, at 8:17 AM, Simon Wilson <[hidden email]> wrote:
> >
> > ostscreen is using (threshold 3):
> >
> >        zen.spamhaus.org*3
> >        bl.mailspike.net*2
> >        b.barracudacentral.org*2
> >        bl.spameatingmonkey.net
> >        bl.spamcop.net
> >        dnsbl.sorbs.net
> >        hostkarma.junkemailfilter.com=127.0.0.2
> >        hostkarma.junkemailfilter.com=127.0.0.4
> >        hostkarma.junkemailfilter.com=127.0.1.2
> >        psbl.surriel.com
> >        swl.spamhaus.org*-4
> >        list.dnswl.org=127.0.[2..15].0*-2
> >        list.dnswl.org=127.0.[2..15].1*-3
> >        list.dnswl.org=127.0.[2..15].[2..3]*-4
> >        wl.mailspike.net=127.0.0.[17;18]*-1
> >        wl.mailspike.net=127.0.0.[19;20]*-2
> >        hostkarma.junkemailfilter.com=127.0.0.1*-1
>
> You'll likely find that after zen.spamhaus.org and bl.barracudacentral + bl.spamcop.net
> you don't need any other RBLs, as they contribute almost nothing to the effectiveness
> of the filter.  Throw in a single whitelist, and you're done.  I think that the current
> list of RBLs is too large.  Start with a short list, grow with care one at a time if
> needed, and only if effectiveness increases non-trivially without too many FPs.
>
> As for a system that's too slow overall, have you checked whether
> your syslog service might be a bottleneck?  Make sure that log
> writes are not synchronous.  With syslog-ng use "unix-dgram" NOT
> "unix-stream".  I've no experience with systemd's logging, check
> for troubles there if applicable.

Disable synchronous writes, and with system-xxx-d, turn off rate
limiting, at least for mail-related events (so that it won't impose
ratelimits before passing events to rsyslogd).

        Wietse
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Optimising new system and postscreen questions

Simon Wilson-7
> Viktor Dukhovni:
>>
>> > On May 1, 2017, at 8:17 AM, Simon Wilson <[hidden email]> wrote:
>> >
>> > ostscreen is using (threshold 3):
>> >
>> >        zen.spamhaus.org*3
>> >        bl.mailspike.net*2
>> >        b.barracudacentral.org*2
>> >        bl.spameatingmonkey.net
>> >        bl.spamcop.net
>> >        dnsbl.sorbs.net
>> >        hostkarma.junkemailfilter.com=127.0.0.2
>> >        hostkarma.junkemailfilter.com=127.0.0.4
>> >        hostkarma.junkemailfilter.com=127.0.1.2
>> >        psbl.surriel.com
>> >        swl.spamhaus.org*-4
>> >        list.dnswl.org=127.0.[2..15].0*-2
>> >        list.dnswl.org=127.0.[2..15].1*-3
>> >        list.dnswl.org=127.0.[2..15].[2..3]*-4
>> >        wl.mailspike.net=127.0.0.[17;18]*-1
>> >        wl.mailspike.net=127.0.0.[19;20]*-2
>> >        hostkarma.junkemailfilter.com=127.0.0.1*-1
>>
>> You'll likely find that after zen.spamhaus.org and  
>> bl.barracudacentral + bl.spamcop.net
>> you don't need any other RBLs, as they contribute almost nothing to  
>> the effectiveness
>> of the filter.  Throw in a single whitelist, and you're done.  I  
>> think that the current
>> list of RBLs is too large.  Start with a short list, grow with care  
>> one at a time if
>> needed, and only if effectiveness increases non-trivially without  
>> too many FPs.
>>
>> As for a system that's too slow overall, have you checked whether
>> your syslog service might be a bottleneck?  Make sure that log
>> writes are not synchronous.  With syslog-ng use "unix-dgram" NOT
>> "unix-stream".  I've no experience with systemd's logging, check
>> for troubles there if applicable.
>
> Disable synchronous writes, and with system-xxx-d, turn off rate
> limiting, at least for mail-related events (so that it won't impose
> ratelimits before passing events to rsyslogd).
>
> Wietse

Thanks gents.

Synchronous writes are already disabled for maillog on the CentOS 7  
server, I hadn't changed it so it must be default.

Rate limits - I'm not getting rate limit messages about dropped log  
entries for either journal or rsyslog.

The server seems quick for most things, and is lightly loaded 99% of  
the time. Heaps of RAM and CPU capacity. I'll tweak the BLs in  
postscreen per Viktor's comments and see how it goes.

I've got a separate email in to the Spamassassin list about slow  
lookups there, and I think it's just the combination of a few things  
that can be tweaked or better understood (including my  
misunderstanding about the postscreen 6 seconds) that makes the server  
seem slow.

Can anyone comment on the value / no value of having zen.spamhaus as  
an RBL in smtpd in addition to it being used by postscreen?

Simon.

--
Simon Wilson
M: 0400 12 11 16

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Optimising new system and postscreen questions

Viktor Dukhovni

> On May 1, 2017, at 10:28 AM, Simon Wilson <[hidden email]> wrote:
>
> Can anyone comment on the value / no value of having zen.spamhaus as an RBL in smtpd in addition to it being used by postscreen?

Keep both.  If you have SpamAssassin doing RBL lookups, raise the
concurrency limit of the filter transport.  That said, keep the
list of RBLs used by SA reasonably trim or disable remote lookups
entirely.

Look at the delays=a/b/c/d data in the delivery logs both pre
and post-filter.  Are the delays high pre-filter (deliveries to
the 127.0.0.1 filter port)?  Are the delays high post-filter?

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Optimising new system and postscreen questions

allenc
In reply to this post by Simon Wilson-7

On 01/05/17 13:17, Simon Wilson wrote:
>
> 3. Any other ways to speed it up, or should I accept the trade-off
> between speed and accuracy of result?
>

If you can create a postscreen white-list of your "regular" remote
hosts,  they will be almost instantly passed on to the mail server.

Hope this helps

Regards

Allen C


Loading...