Postfix snapshot 20091008 with postscreen

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Postfix snapshot 20091008 with postscreen

Wietse Venema
Postfix snapshot 20091008 includes an updated version of the
postscreen daemon. This means it is no longer limited to the
non-production releases.

To make postscreen safe to deploy, it has a permanent whitelist
(default: $mynetworks) that avoids running SMTP protocol tests on
broken network appliances. It also has a permanent blacklist for
networks that you never want to talk to.

In the default "observation" mode, postscreen logs bad client
information but does not drop connections, and can be used to
"pre-fetch" DNSBL information in parallel.

In the non-default "enforcement mode", postscreen drops "bad"
clients, and thus off-loads the SMTP daemons. To make it generally
usable I still have to add the dummy SMTP protocol engine that logs
the senders and recipients of rejected connections. Hopefully that
will be in place later in the Postfix 2.7 development cycle.

        Wietse

HISTORY file entries:

20090918

        Bugfix (introduced Postfix 2.3): with Milter RCPT TO replies
        turned off, there was no automatic flush-before-read on
        the smtpd-to-milter stream, because the read was done on
        the cleanup-to-milter stream. Problem reported by Stephen
        Warren.  File: milter/milter8.c.

20091005

        Bugfix: core dump while printing error message for malformed
        %<letter> sequence in LDAP, MySQL or PostgreSQL configuration.
        File: global/db_common.c. Fix by Victor Duchovni.

20091006

        Feature: "postscreen_whitelist_networks = $mynetworks" (the
        default) to avoid problems with buggy SMTP implementations
        in network appliances.  Note: this feature never uses the
        remote SMTP client hostname.  Files: global/addr_match_list.[hc],
        postscreen/postscreen.c.

        Feature: postscreen_blacklist_networks (default: empty) to
        permanently blacklist hosts or networks. Address syntax is
        as with mynetworks. Note: this feature never uses the remote
        SMTP client hostname.  File: postscreen/postscreen.c.

        Feature: postscreen_blacklist_action (default: continue)
        to control what happens with a permanently blacklisted
        client.

20091007

        Feature: hostname-based check_client_{mx,ns}_access,
        check_reverse_client_hostname_{mx,ns}_access (the client
        IP address is not used). Rob Foehl.  Files: smtpd/smtpd_check.c,
        global/mail_params.h, proto/postconf.proto, mantools/postlink.

20091008

        Documentation: restructured the postscreen(8) manpage as
        a sequence of tests. File: postscreen/postscreen.c.
Reply | Threaded
Open this post in threaded view
|

Re: Postfix snapshot 20091008 with postscreen

Wietse Venema
Wietse Venema:
> Postfix snapshot 20091008 includes an updated version of the
> postscreen daemon. This means it is no longer limited to the
> non-production releases.

In case you haven't seen earlier posts on this topic, postscreen
was released first in a number of Postfix non-production snapshots
over the past summer. Below is a summary, taken from the release
notes.

        Wietse

postscreen(8) is a server that is turned off by default.  When
enabled it runs a number of time-consuming checks in parallel for
all incoming SMTP connections, before clients are allowed to talk
to a real Postfix SMTP server.  It detects clients that start
talking too soon, or clients that appear on DNS blocklists, or
clients that hang up without sending any command.

By doing these checks in a single postscreen(8) process, Postfix
can avoid wasting one SMTP server process per connection. A side
benefit of postscreen(8)'s DNSBL lookups is that DNS records are
already cached before the Postfix SMTP server looks them up later.

postscreen(8) maintains a temporary whitelist of positive decisions.
Once an SMTP client is whitelisted, it is immediately forwarded
to a real Postfix SMTP server process without further checking.

By default, the program logs only statistics, and it does not run
any checks on clients in mynetworks (primarily, to avoid problems
with buggy SMTP implementations in network appliances).  The logging
function alone is already useful for research.

postscreen(8) can be configured to drop clients that start talking
too soon, or clients that appear on DNS blocklists. For details,
see the release notes.
Reply | Threaded
Open this post in threaded view
|

Re: Postfix snapshot 20091008 with postscreen

Miguel Di Ciurcio Filho
In reply to this post by Wietse Venema
Wietse Venema wrote:
> Postfix snapshot 20091008 includes an updated version of the
> postscreen daemon. This means it is no longer limited to the
> non-production releases.
>

Nice!

There is a cool feature on OpenBSD's spamd that makes zombies suffer a lot:

-S secs Stutter at greylisted connections for the specified amount of
        seconds, after which the connection is not stuttered at.
        The default is 10; maximum is 90.


-s secs Delay each character sent to the client by the specified amount
        of seconds.  The default is 1; maximum is 10.

http://www.openbsd.org/cgi-bin/man.cgi?query=spamd&sektion=8

Discarding the greylist feature, sending data very slowly makes zombies
suffer and does not eat our bandwidth.

1) Wait X seconds to send the pre-greeting to detect out of order commands
2) If the client has waited accordingly, optionally, send another
"220-text..." greeting line but slowly, like spamd does.
3) If the client is still there, whitelist it for a day.

Another suggestion: rise the default postscreen_greet_wait from 4 to 10
seconds, or even 15 or 20. I've been using smtpd_error_sleep_time=30s
and so far I had no problems for years and it is very effective keeping
dictionary floods away.

With a setup like this I believe greylisting is not that relevant any more.

Great work.

Miguel



signature.asc (564 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Postfix snapshot 20091008 with postscreen

Reinaldo Gil Lima de Carvalho
On Thu, Oct 8, 2009 at 9:00 PM, Miguel Di Ciurcio Filho
<[hidden email]> wrote:
>
> Another suggestion: rise the default postscreen_greet_wait from 4 to 10
> seconds, or even 15 or 20. I've been using smtpd_error_sleep_time=30s
> and so far I had no problems for years and it is very effective keeping
> dictionary floods away.
>

The sleep time grows cpu time consume and established connections.
Enforce no sleep time and a very low hard limit (to drop connection)
has better performace.

> With a setup like this I believe greylisting is not that relevant any more.
>
> Great work.
>
> Miguel
>
>
>



--
Reinaldo de Carvalho
http://korreio.sf.net
http://python-cyrus.sf.net

"Don't try to adapt the software to the way you work, but rather
yourself to the way the software works" (myself)
Reply | Threaded
Open this post in threaded view
|

Re: Postfix snapshot 20091008 with postscreen

Wietse Venema
In reply to this post by Miguel Di Ciurcio Filho
Miguel Di Ciurcio Filho:
> Wietse Venema wrote:
> > Postfix snapshot 20091008 includes an updated version of the
> > postscreen daemon. This means it is no longer limited to the
> > non-production releases.
> >
>
> Nice!
>
> There is a cool feature on OpenBSD's spamd that makes zombies suffer a lot:

Note, I am primarily interested in keeping the bots away from the
real SMTP server. Unlike spamd and other solutions, I am not so
much interested in keeping botnets busy. People who want to do that
can install spamd. It works with pretty much every MTA.

> Discarding the greylist feature, sending data very slowly makes zombies
> suffer and does not eat our bandwidth.
>
> 1) Wait X seconds to send the pre-greeting to detect out of order commands

If the client is a pre-greeter, the sooner I find out the better.
I want to have the option to quickly drop a connection, or to
quickly capture sender/recipient information so that people can
monitor what mail is being blocked (capturing this information is
next on the todo list; this requires a dummy SMTP engine that could
also be used for greylisting, and if people must, for tarpitting).

> Another suggestion: rise the default postscreen_greet_wait from 4 to 10
> seconds, or even 15 or 20. I've been using smtpd_error_sleep_time=30s
> and so far I had no problems for years and it is very effective keeping
> dictionary floods away.
>
> With a setup like this I believe greylisting is not that relevant any more.

You can adjust the pre-greet wait time to 30s if you like, but I
would not consider that a safe default setting for everyone.

You can find early postscreen results at http://www.postfix.org/wip.html

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Postfix snapshot 20091008 with postscreen

Stan Hoeppner
In reply to this post by Wietse Venema
Wietse Venema put forth on 10/8/2009 1:51 PM:
> Postfix snapshot 20091008 includes an updated version of the
> postscreen daemon. This means it is no longer limited to the
> non-production releases.

Does postscreen run one process per connection, allowing balanced
scheduling across cpus/cores, or is it just one process handling all
connections?  If only one process, do you see possible benefit to
pinning its affinity to a single cpu/core in a high traffic
multi-cpu/core MX, and excluding all other processes from that cpu/core?

--
Stan
Reply | Threaded
Open this post in threaded view
|

Re: Postfix snapshot 20091008 with postscreen

Ralf Hildebrandt
* Stan Hoeppner <[hidden email]>:

> Does postscreen run one process per connection, allowing balanced
> scheduling across cpus/cores, or is it just one process handling all
> connections?  If only one process, do you see possible benefit to
> pinning its affinity to a single cpu/core in a high traffic
> multi-cpu/core MX, and excluding all other processes from that cpu/core?

I don't find postscreen to be CPU intensive, so what's the point?
--
Ralf Hildebrandt
  Geschäftsbereich IT | Abteilung Netzwerk
  CharitĂ© - Universitätsmedizin Berlin
  Campus Benjamin Franklin
  Hindenburgdamm 30 | D-12203 Berlin
  Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
  [hidden email] | http://www.charite.de
           
Reply | Threaded
Open this post in threaded view
|

Re: Postfix snapshot 20091008 with postscreen

Miguel Di Ciurcio Filho
In reply to this post by Reinaldo Gil Lima de Carvalho
Reinaldo de Carvalho wrote:
>
> The sleep time grows cpu time consume and established connections.
> Enforce no sleep time and a very low hard limit (to drop connection)
> has better performace.
>

How an almost halt process, doing nothing could possibly consume any
relevant CPU time or bandwidth?

Reply | Threaded
Open this post in threaded view
|

Re: Postfix snapshot 20091008 with postscreen

Reinaldo Gil Lima de Carvalho
On Fri, Oct 9, 2009 at 11:27 AM, Miguel Di Ciurcio Filho
<[hidden email]> wrote:

> Reinaldo de Carvalho wrote:
>>
>> The sleep time grows cpu time consume and established connections.
>> Enforce no sleep time and a very low hard limit (to drop connection)
>> has better performace.
>>
>
> How an almost halt process, doing nothing could possibly consume any
> relevant CPU time or bandwidth?
>

The process scheduler give a time for each process and the processor
context switch is expensive. If you have hundreds of processes (you'll
have because a high sleep time) can consume all resources.

In my opinion, drop the connection and use a tool to block networks on
firewall level is a better approach.

--
Reinaldo de Carvalho
http://korreio.sf.net
http://python-cyrus.sf.net

"Don't try to adapt the software to the way you work, but rather
yourself to the way the software works" (myself)
Reply | Threaded
Open this post in threaded view
|

Re: Postfix snapshot 20091008 with postscreen

Miguel Di Ciurcio Filho
In reply to this post by Wietse Venema
Wietse Venema wrote:
>
> Note, I am primarily interested in keeping the bots away from the
> real SMTP server. Unlike spamd and other solutions, I am not so
> much interested in keeping botnets busy. People who want to do that
> can install spamd. It works with pretty much every MTA.
>

Point taken.

>
> You can adjust the pre-greet wait time to 30s if you like, but I
> would not consider that a safe default setting for everyone.
>
> You can find early postscreen results at http://www.postfix.org/wip.html
>

Very interesting, I didn't know about that page. How about an RSS feed
so we can keep up with the site updates :-D

Regards,

Miguel