iptables based spam prevention

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

iptables based spam prevention

Niclas Arndt
Hi,

Sorry if this is slightly off-topic, but at least a bunch of experts are listening.

I am using Spamhaus (and other methods) and over time I have amassed a list of IP ranges that (according to Spamhaus) shouldn't be sending e-mail at all. One problem is that this list tends to become quite long and another is that I would like to verify it so that I don't eventually block legitimate e-mail.

On the other hand, I would like to place as little a load as possible on Spamhaus.

Here are my questions: Is the iptables approach at all viable in the long run? Is there any non-commercial way to upload a text file containing spamming IP addresses and have it verified for correctness?

Any other related response is of course welcome.

Thanks in advance.

Niclas
Reply | Threaded
Open this post in threaded view
|

Re: iptables based spam prevention

Glenn English

On Aug 25, 2013, at 12:11 PM, Niclas Arndt wrote:

> I am using Spamhaus (and other methods) and over time I have amassed a list of IP ranges that (according to Spamhaus) shouldn't be sending e-mail at all. One problem is that this list tends to become quite long and another is that I would like to verify it so that I don't eventually block legitimate e-mail.
>
> On the other hand, I would like to place as little a load as possible on Spamhaus.
>
> Here are my questions: Is the iptables approach at all viable in the long run?

It seems to work well for me. I have a small number of users, though, so I can pretty safely block things without loosing stuff.

My mail server's iptables INPUT chain jumps to an "SMTP_BLK" chain when the destination is an SMTP port (actually, it splits on TCP/UDP first, but it eventually ends up in a chain for only SMTP hits). It runs through a little whitelisting, then comes to a long list of IPs that are DROPped.

Those never get to Postfix or Spamhaus, so they don't load anything significantly. The downside is that I have to manually enter the IPs and remove them after a while (I have a shell script to help with this, but it's still a PITA daily duty).

Fail2ban helps too, but it blocks IPs completely instead of just SMTP hits. And it blocks after a number of hits. Manually is more accurate...

> Is there any non-commercial way to upload a text file containing spamming IP addresses and have it verified for correctness?

I'm not sure what you mean here. Iptables won't enter an invalid IP. But if you're asking for something that checks whether the IP is active, the only way I know of to do this would involve a lot of parsing whois answers. And that'd be pretty tough.

If you're asking about something to verify that an IP is indeed a spammer, that's called Spamhaus...

As for just loading a list of IPs, that's easy. If your iptables packet filter is split into chains, like mine is, a simple shell script does the job -- that's one of the reasons I made my PF so complex.

--
Glenn English





smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: iptables based spam prevention

Noel Jones-2
In reply to this post by Niclas Arndt
On 8/25/2013 1:11 PM, Niclas Arndt wrote:

> Hi,
>
> Sorry if this is slightly off-topic, but at least a bunch of experts
> are listening.
>
> I am using Spamhaus (and other methods) and over time I have amassed
> a list of IP ranges that (according to Spamhaus) shouldn't be
> sending e-mail at all. One problem is that this list tends to become
> quite long and another is that I would like to verify it so that I
> don't eventually block legitimate e-mail.
>
> On the other hand, I would like to place as little a load as
> possible on Spamhaus.
>
> Here are my questions: Is the iptables approach at all viable in the
> long run? Is there any non-commercial way to upload a text file
> containing spamming IP addresses and have it verified for correctness?
>
> Any other related response is of course welcome.

I use a postfix check_client_access table that I use as a
whitelist/blacklist before the spamhaus lookup and other anti-spam
checks.

A hash (or even better, cdb) table can grow to millions of entries
before it will place any significant load on postfix, so table size
isn't really a concern.

Since false positives are always concern, doing the rejects in
postfix lets me see the sender and recipient before the mail is
rejected.

Something like:
# main.cf
smtpd_recipient_restrictions =
  permit_mynetworks
  reject_unauth_destination
  check_client_access hash:/etc/postfix/whitelist-blacklist
  ... other anti-spam checks ...




  -- Noel Jones
Reply | Threaded
Open this post in threaded view
|

Re: iptables based spam prevention

@lbutlr
In reply to this post by Niclas Arndt

On 25 Aug 2013, at 12:11 , Niclas Arndt <[hidden email]> wrote:

> Here are my questions: Is the iptables approach at all viable in the long run?

No. This is why RBLS use DNS, because DNS is cheap and it caches automatically. If you are blocking a few sites, (even a few thousand) that is one thing, but when you are trying to block millions? That is something else. Do you want IPTables to have millions of IPs?

--
and I lift my glass to the Awful Truth / which you can't reveal to the
Ears of Youth / except to say it isn't worth a dime

Reply | Threaded
Open this post in threaded view
|

Re: iptables based spam prevention

DTNX Postmaster
In reply to this post by Niclas Arndt
On Aug 26, 2013, at 06:20, [hidden email] wrote:

> On Aug/25.20:11:49, Niclas Arndt wrote:
>> Here are my questions: Is the iptables approach at all viable in the long run? Is there any non-commercial way to upload a text file containing spamming IP addresses and have it verified for correctness?
>
> Your IP tables will get scary quite rapidly, possibly without bounds.
> More so if you do not expire old records.
>
> The XBL component alone should make IP tables faint.
>
> DNS is almost certainly a saner way. In your case, shove your records in
> your own local DNS server and make a private block list. If you have a fit
> of insanity, allow other people to query it too...

Set up your own private rbldnsd; forward requests to it via your local
caching resolver. It scales easily to millions of entries, it does not
require a reload of Postfix for updates, the file format is very simple
and therefore easy to build automatically, and so on.

Also, you get the benefit of the RBL features of Postfix. You can have
different replies for different types of IP addresses; have one type be
rejected directly by postscreen and another type by your recipient
restrictions, for example.

Once you have it running it is trivial to add domain based rejections
as well, and reject HELO hostnames, reverse DNS results, sender domains
and such.

HTH,
Joni

Reply | Threaded
Open this post in threaded view
|

Re: iptables based spam prevention

Jeroen Geilman
In reply to this post by Niclas Arndt
On 08/25/2013 08:11 PM, Niclas Arndt wrote:
Hi,

Sorry if this is slightly off-topic, but at least a bunch of experts are listening.

I am using Spamhaus (and other methods) and over time I have amassed a list of IP ranges that (according to Spamhaus) shouldn't be sending e-mail at all. One problem is that this list tends to become quite long and another is that I would like to verify it so that I don't eventually block legitimate e-mail.

On the other hand, I would like to place as little a load as possible on Spamhaus.

Here are my questions: Is the iptables approach at all viable in the long run? Is there any non-commercial way to upload a text file containing spamming IP addresses and have it verified for correctness?

postfix 2.8 and later offer the postscreen(8) triage service, which deals very efficiently with large amounts of DNSBL lookups.
Run a local DNS cache on the postfix machine and point postscreen at zen.
You'll be hitting the spamhaus non-commercial limit long before you hit the local cache's limits.

This automatically adds and expires DNSBL entries without any effort from you, as a free bonus (this is the biggest problem with your iptables approach.)

-- 
J.
Reply | Threaded
Open this post in threaded view
|

Re: iptables based spam prevention

Stan Hoeppner
On 8/27/2013 5:01 PM, Jeroen Geilman wrote:
> On 08/25/2013 08:11 PM, Niclas Arndt wrote:

>> Sorry if this is slightly off-topic, but at least a bunch of experts
>> are listening.
>>
>> I am using Spamhaus (and other methods) and over time I have amassed a
>> list of IP ranges that (according to Spamhaus) shouldn't be sending
>> e-mail at all.

Given your description this would be the PBL list, typically filled with
consumer broadband IP ranges.

>> One problem is that this list tends to become quite
>> long and another is that I would like to verify it so that I don't
>> eventually block legitimate e-mail.

This suggests you've collected these IPs from your logs and have built a
local table.  Why?  I don't understand what you mean by "verify it".
WRT blocking legitimate email that's always a risk with dnsbls, though
in the case of the Spamhaus lists a very low risk.

>> On the other hand, I would like to place as little a load as possible
>> on Spamhaus.

See below.  But note you are allowed 300K queries/day to Zen.

>> Here are my questions: Is the iptables approach at all viable in the
>> long run? Is there any non-commercial way to upload a text file
>> containing spamming IP addresses and have it verified for correctness?
>
> postfix 2.8 and later offer the postscreen(8) triage service, which
> deals very efficiently with large amounts of DNSBL lookups.
> Run a local DNS cache on the postfix machine and point postscreen at zen.
> You'll be hitting the spamhaus non-commercial limit long before you hit
> the local cache's limits.
>
> This automatically adds and expires DNSBL entries without any effort
> from you, as a free bonus (this is the biggest problem with your
> iptables approach.)

A local caching resolver is a good start.  I use pdns_recursor.

If you're concerned about your query load to Spamhaus public Zen servers
then block spam connections without using dnsbls, or more precisely,
block as much as possible before querying dnsbls.  This is what myself
and many others do.  I.e. use the least expensive tools first and the
most expensive last.  Expense being defined as time to completion and
resources consumed (net/CPU/RAM/disk IO).

This not only decreases the load you put on dnsbls, but it also
increases throughput due to decreasing remote lookup latency per
message.  Judicious use of inbuilt Postfix features and local tables can
cut per msg latency from a few dozen milliseconds down to a few
microseconds.

To do this, enable Postcreen in its standard mode.  Do not configure
dnsbl block lists in postscreen.  This will stop nearly all bot traffic.
 Next, use the "everything under smtpd_recipient_restrictions" main.cf
model so you can precisely control the order in which restrictions are
processed.  Inbuilt Postfix restrictions are the least expensive, then
CDB, hash, and CIDR tables.  REGEXP and PCRE tables are a little more
expensive, possibly much more if tables are large (and without loops)
and expressions complex.  Header checks are normally next in expense
followed by body checks, and local policy daemons and content filters
can be more expensive still.

So as a general starting point you'd want something similar to this.
This configuration assumes the Postfix server is an MX as well as a
submission relay.

smtpd_recipient_restrictions =

# These are inbuilt Postfix restrictions

        permit_mynetworks
        permit_sasl_authenticated
        reject_unauth_destination
        reject_unknown_reverse_client_hostname
        reject_non_fqdn_sender
        reject_non_fqdn_helo_hostname
        reject_invalid_helo_hostname
        reject_unlisted_recipient

# These check a hash table of domains which have sent spam here
# or that have been obtained via intelligence (A/S mailing lists)
# and check dnswl.org for sender that should be allowed through

        check_reverse_client_hostname_access hash:/etc/postfix/blacklist
        check_helo_access hash:/etc/postfix/blacklist
        check_sender_access hash:/etc/postfix/blacklist
        check_client_access hash:/etc/postfix/whitelist
        permit_dnswl_client list.dnswl.org=127.0.[2..14].[2..3]

# This checks a rather larger CIDR table of snowshoe spammer netblocks
# that I've been building for ~4 years now

        check_client_access /etc/postfix/spammer

# If none of the previous restrictions reject the spam I then query
# various [dns|rhs]bls.

        reject_rbl_client zen.spamhaus.org
        reject_rbl_client b.barracudacentral.org
        reject_rbl_client psbl.surriel.com
        reject_rhsbl_reverse_client dbl.spamhaus.org
        reject_rhsbl_sender dbl.spamhaus.org
        reject_rhsbl_helo dbl.spamhaus.org

If you want to be able to score [dns|rhs]bl results you can use a policy
filter here such as postfwd or policyd instead of the simple
reject/allow offered by these Postfix restrictions.

Combining this with a local caching DNS resolver as Jeroen suggested you
should cut down on your dnsbl query load pretty substantially.

Now, regarding the list you've already built, if you would like I can
provide you a tool that will automatically convert it into a Postfix
CIDR table that you can use with check_client_access.  Though if you
built that list in the manner I'm guessing, then I'd recommend against
it.  Some of the IPs may be legit systems that were temporarily listed
due to malware outbreak or similar, and are permanently in your file.

--
Stan