problems with pcre checks

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

problems with pcre checks

Stan Hoeppner
Hi all,

I just had a viagra spam from gprs4f7a24e6.pool.t-umts.hu
(gprs4f7a24e6.pool.t-umts.hu [79.122.36.230] sneak past both of my pcre
checks which should have killed it.  Until today they've been working
flawlessly, or so I believe, and I have ample log entries showing
they've been working.  Here are my checks, both performed via
smtpd_client_restrictions but without explicit check_client_access:

smtpd_client_restrictions =
        pcre:/etc/postfix/access.pcre,
        pcre:/etc/postfix/check_client_fqdn.pcre

/etc/postfix/access.pcre
/^.*?(lv|ec|id|ph|at|hu|tr|ee|pl|ro|my|co|tw|br|za|do|cz|bg|by|kr|jp|fr|cn|ru)$/i
550 We do not accept mail from .$1 domains

/etc/postfix/check_client_fqdn.pcre
/\.?(dhcp|dialup|dynamic|ppp|pool)\.?/       REJECT
Dynamic/DSL/Residential not allowed
/\.(dsl|\d+dsl|dsl\d+)\./                    REJECT
Dynamic/DSL/Residential not allowed

Logs show both killing spam yesterday.  check_client_fqdn.pcre has
already killed some today, but nothing killed by access.pcre yet today.

Any idea what's gone awry?  Is there a way I can test the offending
FQrDNS against these two filters from the command line, with postconf or
something?  What log data should I be looking for that may tell me why
pcre checks suddenly stopped working, or just didn't work on this
Hungrian Viagra connection?

Disclaimer:  Please refrain from subjective comments WRT my blocking of
FQrDNS TLDs.  I run a very small domain in the U.S. that will very
likely never receive legit mail from those countries, but has received
spam from them.  It's a quick, easy, and effective way to block spam.
It may not be "PC" or to your liking, but please accept the fact that it
works for me.

Thanks.

--
Stan
Reply | Threaded
Open this post in threaded view
|

Re: problems with pcre checks

Noel Jones-2
On 10/2/2009 10:05 AM, Stan Hoeppner wrote:

> Hi all,
>
> I just had a viagra spam from gprs4f7a24e6.pool.t-umts.hu
> (gprs4f7a24e6.pool.t-umts.hu [79.122.36.230] sneak past both of my pcre
> checks which should have killed it.  Until today they've been working
> flawlessly, or so I believe, and I have ample log entries showing
> they've been working.  Here are my checks, both performed via
> smtpd_client_restrictions but without explicit check_client_access:
>
> smtpd_client_restrictions =
>          pcre:/etc/postfix/access.pcre,
>          pcre:/etc/postfix/check_client_fqdn.pcre

Better to use the proper syntax (although that's not the problem).

smtpd_client_restrictions =
   check_client_access pcre:/etc/postfix/access.pcre
   check_client_access pcre:/etc/postfix/check_client_fqdn.pcre

Any good excuse why these can't be combined into one pcre file?

>
> /etc/postfix/access.pcre
> /^.*?(lv|ec|id|ph|at|hu|tr|ee|pl|ro|my|co|tw|br|za|do|cz|bg|by|kr|jp|fr|cn|ru)$/i
> 550 We do not accept mail from .$1 domains

/\.(lv|ec|...|ru)$/  REJECT ...

Don't specify useless leading wildcards. Don't use the /i flag
unless you really need it - this might be why it didn't match.

And both your maps would be more effective if you use them
with check_reverse_client_hostname_access rather than
check_client_access - this might also be why it didn't match.
  Requires postfix 2.6 or newer.
http://www.postfix.org/postconf.5.html#check_reverse_client_hostname_access

>
> /etc/postfix/check_client_fqdn.pcre
> /\.?(dhcp|dialup|dynamic|ppp|pool)\.?/       REJECT
> Dynamic/DSL/Residential not allowed
> /\.(dsl|\d+dsl|dsl\d+)\./                    REJECT
> Dynamic/DSL/Residential not allowed
>
> Logs show both killing spam yesterday.  check_client_fqdn.pcre has
> already killed some today, but nothing killed by access.pcre yet today.
>
> Any idea what's gone awry?  Is there a way I can test the offending
> FQrDNS against these two filters from the command line, with postconf or
> something?  What log data should I be looking for that may tell me why
> pcre checks suddenly stopped working, or just didn't work on this
> Hungrian Viagra connection?

test maps with postmap -q
http://www.postfix.org/postmap.1.html

# postmap -q key pcre:/etc/postfix/access.pcre

The key postfix used is in the mail log.

   -- Noel Jones
Reply | Threaded
Open this post in threaded view
|

problems with pcre checks

Stan Hoeppner
Noel Jones put forth on 10/2/2009 10:54 AM:

> On 10/2/2009 10:05 AM, Stan Hoeppner wrote:
>> Hi all,
>>
>> I just had a viagra spam from gprs4f7a24e6.pool.t-umts.hu
>> (gprs4f7a24e6.pool.t-umts.hu [79.122.36.230] sneak past both of my pcre
>> checks which should have killed it.  Until today they've been working
>> flawlessly, or so I believe, and I have ample log entries showing
>> they've been working.  Here are my checks, both performed via
>> smtpd_client_restrictions but without explicit check_client_access:
>>
>> smtpd_client_restrictions =
>>          pcre:/etc/postfix/access.pcre,
>>          pcre:/etc/postfix/check_client_fqdn.pcre
>
> Better to use the proper syntax (although that's not the problem).

Acknowledged.

> smtpd_client_restrictions =
>   check_client_access pcre:/etc/postfix/access.pcre
>   check_client_access pcre:/etc/postfix/check_client_fqdn.pcre
>
> Any good excuse why these can't be combined into one pcre file?

Probably not.  I copy/pasted from examples I found at different points
in time.  I guess it just made sense at the time to start a new file for
the 2nd one.  Good idea though, I'll consolidate them.

>> /etc/postfix/access.pcre
>> /^.*?(lv|ec|id|ph|at|hu|tr|ee|pl|ro|my|co|tw|br|za|do|cz|bg|by|kr|jp|fr|cn|ru)$/i
>>
>> 550 We do not accept mail from .$1 domains
>
> /\.(lv|ec|...|ru)$/  REJECT ...
>
> Don't specify useless leading wildcards. Don't use the /i flag unless
> you really need it - this might be why it didn't match.

I'll read up on regexes and see if I can clean that up.

> And both your maps would be more effective if you use them with
> check_reverse_client_hostname_access rather than check_client_access -
> this might also be why it didn't match.  Requires postfix 2.6 or newer.
> http://www.postfix.org/postconf.5.html#check_reverse_client_hostname_access

I'm on 2.3.8.  I always stick with my distro's packages, so my next rev
will only be up to 2.5.5-1.1 when I upgrade to Lenny (I'm running
Debian).  I've got some issues I need to work out before upgrading to
Lenny though.

>> /etc/postfix/check_client_fqdn.pcre
>> /\.?(dhcp|dialup|dynamic|ppp|pool)\.?/       REJECT
>> Dynamic/DSL/Residential not allowed
>> /\.(dsl|\d+dsl|dsl\d+)\./                    REJECT
>> Dynamic/DSL/Residential not allowed
<snip>>
> test maps with postmap -q
> http://www.postfix.org/postmap.1.html
>
> # postmap -q key pcre:/etc/postfix/access.pcre
>
> The key postfix used is in the mail log.

Thank you for the command Noel.  Using the current pcre's, testing shows:

greer:/etc/postfix# postmap -q gprs4f7a24e6.pool.t-umts.hu
pcre:/etc/postfix/access.pcre
550 We do not accept mail from .hu domains

greer:/etc/postfix# postmap -q gprs4f7a24e6.pool.t-umts.hu
pcre:/etc/postfix/check_client_fqdn.pcre
REJECT Dynamic/DSL/Residential not allowed

Just as I thought.  It should have been working.  Any ideas on why I'd
have a temp failure with pcre checks?  As I said before, this is the
first and only time this has happened.  I've got 74 rejects in the mail
log for check_client_fqdn.pcre, and I've got 112 for access.pcre.  The
most recent reject I have for either is early this morning:

Oct  2 03:35:42 greer postfix/smtpd[23597]: NOQUEUE: reject: RCPT from
94-43-142-94.dsl.utg.ge[94.43.142.94]: 554 5.7.1
<94-43-142-94.dsl.utg.ge[94.43.142.94]>: Client host rejected:
Dynamic/DSL/Residential not allowed; from=<[hidden email]>
to=<[hidden email]> proto=SMTP helo=<94.43.142.94>

so we know at least one pcre check was working as of 3:35am CDT today
Oct 2.  The mail that sneaked through was at 6:39am CDT today Oct 2.

I'll keep an eye on the log throughout the day to see if the PCREs are
working.  My mail flow is small and sporadic, so it may take a bit to
see evidence of what is, or is not, happening.

Has anyone ever head of temporary pcre processing failures?

This issue has really got me stumped...

--
Stan

Reply | Threaded
Open this post in threaded view
|

Re: problems with pcre checks

Noel Jones-2
On 10/2/2009 11:51 AM, Stan Hoeppner wrote:

> Noel Jones put forth on 10/2/2009 10:54 AM:
>> On 10/2/2009 10:05 AM, Stan Hoeppner wrote:
>>> Hi all,
>>>
>>> I just had a viagra spam from gprs4f7a24e6.pool.t-umts.hu
>>> (gprs4f7a24e6.pool.t-umts.hu [79.122.36.230] sneak past both of my pcre
>>> checks which should have killed it.  Until today they've been working
>>> flawlessly, or so I believe, and I have ample log entries showing
>>> they've been working.  Here are my checks, both performed via
>>> smtpd_client_restrictions but without explicit check_client_access:
>>>
>>> smtpd_client_restrictions =
>>>           pcre:/etc/postfix/access.pcre,
>>>           pcre:/etc/postfix/check_client_fqdn.pcre
>>
>> Better to use the proper syntax (although that's not the problem).
>
> Acknowledged.
>
>> smtpd_client_restrictions =
>>    check_client_access pcre:/etc/postfix/access.pcre
>>    check_client_access pcre:/etc/postfix/check_client_fqdn.pcre
>>
>> Any good excuse why these can't be combined into one pcre file?
>
> Probably not.  I copy/pasted from examples I found at different points
> in time.  I guess it just made sense at the time to start a new file for
> the 2nd one.  Good idea though, I'll consolidate them.
>
>>> /etc/postfix/access.pcre
>>> /^.*?(lv|ec|id|ph|at|hu|tr|ee|pl|ro|my|co|tw|br|za|do|cz|bg|by|kr|jp|fr|cn|ru)$/i
>>>
>>> 550 We do not accept mail from .$1 domains
>>
>> /\.(lv|ec|...|ru)$/  REJECT ...
>>
>> Don't specify useless leading wildcards. Don't use the /i flag unless
>> you really need it - this might be why it didn't match.
>
> I'll read up on regexes and see if I can clean that up.
>
>> And both your maps would be more effective if you use them with
>> check_reverse_client_hostname_access rather than check_client_access -
>> this might also be why it didn't match.  Requires postfix 2.6 or newer.
>> http://www.postfix.org/postconf.5.html#check_reverse_client_hostname_access
>
> I'm on 2.3.8.  I always stick with my distro's packages, so my next rev
> will only be up to 2.5.5-1.1 when I upgrade to Lenny (I'm running
> Debian).  I've got some issues I need to work out before upgrading to
> Lenny though.
>
>>> /etc/postfix/check_client_fqdn.pcre
>>> /\.?(dhcp|dialup|dynamic|ppp|pool)\.?/       REJECT
>>> Dynamic/DSL/Residential not allowed
>>> /\.(dsl|\d+dsl|dsl\d+)\./                    REJECT
>>> Dynamic/DSL/Residential not allowed
> <snip>>
>> test maps with postmap -q
>> http://www.postfix.org/postmap.1.html
>>
>> # postmap -q key pcre:/etc/postfix/access.pcre
>>
>> The key postfix used is in the mail log.
>
> Thank you for the command Noel.  Using the current pcre's, testing shows:
>
> greer:/etc/postfix# postmap -q gprs4f7a24e6.pool.t-umts.hu
> pcre:/etc/postfix/access.pcre
> 550 We do not accept mail from .hu domains
>
> greer:/etc/postfix# postmap -q gprs4f7a24e6.pool.t-umts.hu
> pcre:/etc/postfix/check_client_fqdn.pcre
> REJECT Dynamic/DSL/Residential not allowed
>
> Just as I thought.  It should have been working.  Any ideas on why I'd
> have a temp failure with pcre checks?  As I said before, this is the
> first and only time this has happened.  I've got 74 rejects in the mail
> log for check_client_fqdn.pcre, and I've got 112 for access.pcre.  The
> most recent reject I have for either is early this morning:
>
> Oct  2 03:35:42 greer postfix/smtpd[23597]: NOQUEUE: reject: RCPT from
> 94-43-142-94.dsl.utg.ge[94.43.142.94]: 554 5.7.1
> <94-43-142-94.dsl.utg.ge[94.43.142.94]>: Client host rejected:
> Dynamic/DSL/Residential not allowed; from=<[hidden email]>
> to=<[hidden email]>  proto=SMTP helo=<94.43.142.94>
>
> so we know at least one pcre check was working as of 3:35am CDT today
> Oct 2.  The mail that sneaked through was at 6:39am CDT today Oct 2.
>
> I'll keep an eye on the log throughout the day to see if the PCREs are
> working.  My mail flow is small and sporadic, so it may take a bit to
> see evidence of what is, or is not, happening.
>
> Has anyone ever head of temporary pcre processing failures?
>
> This issue has really got me stumped...
>
> --
> Stan
>


Was the mail addressed to postmaster?  The postmaster address
gets a free ride in some versions of postfix.  To disable that
loophole, set in main.cf:
address_verify_sender = $double_bounce_sender

If you need further help, show "postconf -n" output, logging
of the offending mail passing through postfix, and headers
from the mail.


   -- Noel Jones

Reply | Threaded
Open this post in threaded view
|

problems with pcre checks

Stan Hoeppner
Noel Jones put forth on 10/2/2009 12:00 PM:

> Was the mail addressed to postmaster?  The postmaster address gets a
> free ride in some versions of postfix.

No, it wasn't addressed to Postmaster, but I wish you'd have said this
before I made a fool of myself, because your suggestion here jolted my
thought process.  It got through because of a "free ride".  I manage a
small spam trap, and I like to keep an eye on what's landing in it.  The
trap address is unfiltered (you may have actually been the one who
helped me get that trap address whitelisted properly a couple weeks ago
Noel).  Anyway, I have Postfix route that spam into my mailbox, and I
have T-Bird sort it into the trap folder.  You guessed it...

My T-Bird spam trap filter rule is based on the "To" address, and this
spam was addressed to "undisclosed-recipients", but with a RCPT TO of
the trap address.  Thus, the T-Bird rule couldn't sort it, and when it
showed up in my main inbox folder, I just assumed it was sent to
[hidden email].  I've _never_ had a T-Bird sorting problem until
today, which is why the thought hadn't yet popped into my head that this
issue could be MUA related.

> If you need further help, show "postconf -n" output, logging of the
> offending mail passing through postfix, and headers from the mail.

The only additional help I probably need is a good kick in the butt or a
bucket of water thrown on my head. ;)  Sorry I dragged you through this,
but I'm sure glad you were along for the ride.  You unknowingly solved
my issue with two simple words:  "free ride".

I'll be very mindful of today's experience moving forward.

Thank Noel.

--
Stan
Reply | Threaded
Open this post in threaded view
|

Re: problems with pcre checks

mouss-4
Stan Hoeppner a écrit :

> Noel Jones put forth on 10/2/2009 12:00 PM:
>
>> Was the mail addressed to postmaster?  The postmaster address gets a
>> free ride in some versions of postfix.
>
> No, it wasn't addressed to Postmaster, but I wish you'd have said this
> before I made a fool of myself, because your suggestion here jolted my
> thought process.  It got through because of a "free ride".  I manage a
> small spam trap, and I like to keep an eye on what's landing in it.  The
> trap address is unfiltered (you may have actually been the one who
> helped me get that trap address whitelisted properly a couple weeks ago
> Noel).  Anyway, I have Postfix route that spam into my mailbox, and I
> have T-Bird sort it into the trap folder.  You guessed it...
>
> My T-Bird spam trap filter rule is based on the "To" address, and this
> spam was addressed to "undisclosed-recipients", but with a RCPT TO of
> the trap address.  Thus, the T-Bird rule couldn't sort it, and when it
> showed up in my main inbox folder, I just assumed it was sent to
> [hidden email].  I've _never_ had a T-Bird sorting problem until
> today, which is why the thought hadn't yet popped into my head that this
> issue could be MUA related.
>

instead of relying on To, use recipient delimiter to route the tapped
mail to say stan+spam@..., then have TB use the Delivered-To header that
is added by postfix. of course, you can do this at delivery time (sieve
if dovecot, maildrop rules if courier, ... etc).


>> If you need further help, show "postconf -n" output, logging of the
>> offending mail passing through postfix, and headers from the mail.
>
> The only additional help I probably need is a good kick in the butt or a
> bucket of water thrown on my head. ;)  Sorry I dragged you through this,
> but I'm sure glad you were along for the ride.  You unknowingly solved
> my issue with two simple words:  "free ride".
>
> I'll be very mindful of today's experience moving forward.
>
> Thank Noel.
>
> --
> Stan

Reply | Threaded
Open this post in threaded view
|

problems with pcre checks

Stan Hoeppner
mouss put forth on 10/3/2009 8:32 AM:

> instead of relying on To, use recipient delimiter to route the tapped
> mail to say stan+spam@..., then have TB use the Delivered-To header that
> is added by postfix. of course, you can do this at delivery time (sieve
> if dovecot, maildrop rules if courier, ... etc).

I just reconfigured the rule to filter on 'to <[hidden email]>'
being in the received headers.  That should sufficiently fix the issue
for now.  I'm using recipient_bcc_maps to send copies of all these trap
messages to a, umm, organization, and I need to keep the original
headers intact.

Thanks for the suggestion though mouss.  I may implement something like
that down the road.

--
Stan