matching IP ranges in headers

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

matching IP ranges in headers

Louis-David Mitterrand-21
Hi,

A lot of spam comes from certain ip ranges (e.g. west africa) through
relays (large ISPs) that would be too onerous to block. To filter these
I am presently matching:

        /^((Received|X-Originating-IP):.+\b(124\.120\.1\.(<IP RANGE IN REGEX>)\b/

in pcre:/etc/postfix/header_access. But converting IP ranges to regex'es
is time consuming and error prone.

Is there a way to use a cidr table for header matching while retaining
control of the prefix ^(Received|X-Originating-IP) ?

Or another better way?

Thanks,
Reply | Threaded
Open this post in threaded view
|

Re: matching IP ranges in headers

Barney Desmond
2009/6/25 Louis-David Mitterrand <[hidden email]>:
>        /^((Received|X-Originating-IP):.+\b(124\.120\.1\.(<IP RANGE IN REGEX>)\b/
>
> in pcre:/etc/postfix/header_access. But converting IP ranges to regex'es
> is time consuming and error prone.
>
> Is there a way to use a cidr table for header matching while retaining
> control of the prefix ^(Received|X-Originating-IP) ?

I suspect you're out of luck. To be sufficiently general to be useful,
I understand Postfix takes the entire header and uses it as a lookup
key to the defined table (you're not limited to pcre/regexp, but not
much else will be useful). Bare IP addresses of course wouldn't appear
as headers, so you can't use a CIDR table.

> Or another better way?

Someone else may have a better, or they might say you should be using
a DNSBL for this sort of purpose. I personally suspect this IP-based
method won't stay up-to-date enough for practical purposes, which is
why it's probably easier to let someone else do the work and use their
blacklist.
Reply | Threaded
Open this post in threaded view
|

Re: matching IP ranges in headers

Henrik K
On Thu, Jun 25, 2009 at 10:14:29PM +1000, Barney Desmond wrote:

> 2009/6/25 Louis-David Mitterrand <[hidden email]>:
> >        /^((Received|X-Originating-IP):.+\b(124\.120\.1\.(<IP RANGE IN REGEX>)\b/
> >
> > in pcre:/etc/postfix/header_access. But converting IP ranges to regex'es
> > is time consuming and error prone.
> >
> > Is there a way to use a cidr table for header matching while retaining
> > control of the prefix ^(Received|X-Originating-IP) ?
>
> I suspect you're out of luck. To be sufficiently general to be useful,
> I understand Postfix takes the entire header and uses it as a lookup
> key to the defined table (you're not limited to pcre/regexp, but not
> much else will be useful). Bare IP addresses of course wouldn't appear
> as headers, so you can't use a CIDR table.
>
> > Or another better way?
>
> Someone else may have a better, or they might say you should be using
> a DNSBL for this sort of purpose. I personally suspect this IP-based
> method won't stay up-to-date enough for practical purposes, which is
> why it's probably easier to let someone else do the work and use their
> blacklist.

If you have a cidr list, it's sufficiently easy to use some perl magic and
make regex out of it. You can make pretty large lists into smallish regex
since there's only so many characters in IPs. I already made a bare script
our of curiosity, need to finish it..

Reply | Threaded
Open this post in threaded view
|

Re: matching IP ranges in headers

mouss-4
In reply to this post by Louis-David Mitterrand-21
Louis-David Mitterrand a écrit :

> Hi,
>
> A lot of spam comes from certain ip ranges (e.g. west africa) through
> relays (large ISPs) that would be too onerous to block. To filter these
> I am presently matching:
>
> /^((Received|X-Originating-IP):.+\b(124\.120\.1\.(<IP RANGE IN REGEX>)\b/
>
> in pcre:/etc/postfix/header_access. But converting IP ranges to regex'es
> is time consuming and error prone.





except for simple cases, it is impossible.

>
> Is there a way to use a cidr table for header matching while retaining
> control of the prefix ^(Received|X-Originating-IP) ?
>

you need a proxy_filter or a milter. or a content_filter if it is ok to
tag/:quarantine/... instead of reject.

in my SA, I have things like these:


# Nigeria
header COUNTRY_NG X-Relay-Countries=~/\bNG\b/
describe COUNTRY_NG Relayed via Nigeria

# Cote d'Ivoire
header COUNTRY_CI X-Relay-Countries=~/\bCI\b/
describe COUNTRY_CI Relayed via Cote-d-Ivoire

...



> Or another better way?
>
> Thanks,

Reply | Threaded
Open this post in threaded view
|

Re: matching IP ranges in headers

Victor Duchovni
On Sat, Jun 27, 2009 at 12:00:09AM +0200, mouss wrote:

> Louis-David Mitterrand a ?crit :
> > Hi,
> >
> > A lot of spam comes from certain ip ranges (e.g. west africa) through
> > relays (large ISPs) that would be too onerous to block. To filter these
> > I am presently matching:
> >
> > /^((Received|X-Originating-IP):.+\b(124\.120\.1\.(<IP RANGE IN REGEX>)\b/
> > in pcre:/etc/postfix/header_access. But converting IP ranges to regex'es
> > is time consuming and error prone.
>
> except for simple cases, it is impossible.

Impossible is too strong. Just painful if done by hand.

--
        Viktor.

Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.

To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:[hidden email]?body=unsubscribe%20postfix-users>

If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.
Reply | Threaded
Open this post in threaded view
|

Re: matching IP ranges in headers

mouss-4
Victor Duchovni a écrit :

> On Sat, Jun 27, 2009 at 12:00:09AM +0200, mouss wrote:
>
>> Louis-David Mitterrand a ?crit :
>>> Hi,
>>>
>>> A lot of spam comes from certain ip ranges (e.g. west africa) through
>>> relays (large ISPs) that would be too onerous to block. To filter these
>>> I am presently matching:
>>>
>>> /^((Received|X-Originating-IP):.+\b(124\.120\.1\.(<IP RANGE IN REGEX>)\b/
>>> in pcre:/etc/postfix/header_access. But converting IP ranges to regex'es
>>> is time consuming and error prone.
>> except for simple cases, it is impossible.
>
> Impossible is too strong. Just painful if done by hand.
>

true. I meant there is no "simple" correspondence between cidr and
regular expressions. that is, one can't do
        /....124\.120\.1\64\/28 .../

now, one nice feature would be the ability to pass some parts to a check:

<INVALID> (those reading this should be aware that the following is
fiction. it doesn't exist. so don't ask how you could make it work on
your system).

/^Received: ..... \[(\d\.]+)\] .../ check_ip $1

check_ip =
        reject_unknown_client{$arg}
        reject_rbl_client zen.spamhaus.org{$arg}
        ...
</INVALID>

but then again, we come back to "what would be generic control grammar?"
which is not a simple problem.

Reply | Threaded
Open this post in threaded view
|

Re: matching IP ranges in headers

Erwan David
Le Sat 27/06/2009, mouss disait
>
> but then again, we come back to "what would be generic control grammar?"
> which is not a simple problem.
>

I think this would lead to embedding a full scripting language, like some
applications embed lua...

Not sure it woud be a good thing.

--
Erwan
Reply | Threaded
Open this post in threaded view
|

Re: matching IP ranges in headers

Henrik K
In reply to this post by mouss-4
On Sat, Jun 27, 2009 at 12:27:50AM +0200, mouss wrote:

> Victor Duchovni a écrit :
> > On Sat, Jun 27, 2009 at 12:00:09AM +0200, mouss wrote:
> >
> >> Louis-David Mitterrand a ?crit :
> >>> Hi,
> >>>
> >>> A lot of spam comes from certain ip ranges (e.g. west africa) through
> >>> relays (large ISPs) that would be too onerous to block. To filter these
> >>> I am presently matching:
> >>>
> >>> /^((Received|X-Originating-IP):.+\b(124\.120\.1\.(<IP RANGE IN REGEX>)\b/
> >>> in pcre:/etc/postfix/header_access. But converting IP ranges to regex'es
> >>> is time consuming and error prone.
> >> except for simple cases, it is impossible.
> >
> > Impossible is too strong. Just painful if done by hand.
> >
>
> true. I meant there is no "simple" correspondence between cidr and
> regular expressions. that is, one can't do
> /....124\.120\.1\64\/28 .../

$ ./cidr_to_regex.pl
124.120.1.64/28
10.1.1.1 - 10.1.12.255
->
1(?:0\.1\.(?:[23456789]|1[012]?)\.\d{1,3}|24\.120\.1\.(?:6[456789]|7\d))

It's pretty simple when you think about it. Just convert the CIDR to say
full C-classes (resulting in a.b.c.\d+{1,3}), only the last one might need
special stuff. Full B-classes to C-classes etc.. unless you have a very big
list, the resulting regex size will not be that huge since many of the
prefixes will be common.

Reply | Threaded
Open this post in threaded view
|

Re: matching IP ranges in headers

Byung-Hee HWANG
In reply to this post by Louis-David Mitterrand-21
Louis-David Mitterrand <[hidden email]> writes:

> Hi,
>
> A lot of spam comes from certain ip ranges (e.g. west africa) through
> relays (large ISPs) that would be too onerous to block. To filter these
> I am presently matching:
>
> /^((Received|X-Originating-IP):.+\b(124\.120\.1\.(<IP RANGE IN REGEX>)\b/
>
> in pcre:/etc/postfix/header_access. But converting IP ranges to regex'es
> is time consuming and error prone.
>
> Is there a way to use a cidr table for header matching while retaining
> control of the prefix ^(Received|X-Originating-IP) ?
>
> Or another better way?

Use Google Apps: http://www.google.com/a ;;
Unfortunately, Google Apps is the best solution for spam filtering, as
far as i know.

Sincerely,

--
Byung-Hee HWANG, KNU
∑ WWW: http://izb.knu.ac.kr/~bh/