Header check and script

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Header check and script

Richard Smits
Hello,

I have a puzzle which I hope someone can help me with. I want to do a
header check with postfix, and if the subject, contains a word, call a
script or program to put the mail message in a quarantaine.

I also want to make an exception for some email recipients.

I know we can do this with amavis and spamassassin, but I don't want to
use spamassassin. The reason for this construction is that our isp will
implement a spamfilter, and i don't want to check the emails twice.

If the message has a spamtag in the subject, i want to put it in a
quarantainedir. But some people want to receive their spam, so those are
the exceptions.

So, is this possible with postfix alone, and without spamassassin ?

Any hints are welcome....

Thanks...

Richard Smits
Reply | Threaded
Open this post in threaded view
|

Re: Header check and script

Brian Evans - Postfix List
Richard Smits wrote:

> Hello,
>
> I have a puzzle which I hope someone can help me with. I want to do a
> header check with postfix, and if the subject, contains a word, call a
> script or program to put the mail message in a quarantaine.
>
> I also want to make an exception for some email recipients.
>
> I know we can do this with amavis and spamassassin, but I don't want
> to use spamassassin. The reason for this construction is that our isp
> will implement a spamfilter, and i don't want to check the emails twice.
>
> If the message has a spamtag in the subject, i want to put it in a
> quarantainedir. But some people want to receive their spam, so those
> are the exceptions.
>
> So, is this possible with postfix alone, and without spamassassin ?
No.  You need some sort of content filter daemon to do this.
Reasoning:

   1. header_checks can only REDIRECT to another address (which of
      course can then be piped) or FILTER to a content filter
   2. header_checks CANNOT exempt certain recipients for a given
      transport (assuming you put it in master.cf and not main.cf)

If you remove the exemption requirement, then it IS possible for postfix
alone if and only if a previous filter applied such a header.  Then, the
REDIRECT option becomes available.

Brian
Reply | Threaded
Open this post in threaded view
|

Re: Header check and script

mouss-2
In reply to this post by Richard Smits
Richard Smits wrote:
> Hello,
>
> I have a puzzle which I hope someone can help me with. I want to do a
> header check with postfix, and if the subject, contains a word, call a
> script or program to put the mail message in a quarantaine.
>

postfix can't run a program, but you can HOLD, DISCARD, FILTER, ...
based on the contents of _one_ header or body line.


> I also want to make an exception for some email recipients.

postfix header and body checks have no exception mechanism. if you want
exceptions, you need a content filter.

>
> I know we can do this with amavis and spamassassin, but I don't want to
> use spamassassin. The reason for this construction is that our isp will
> implement a spamfilter, and i don't want to check the emails twice.
>
> If the message has a spamtag in the subject, i want to put it in a
> quarantainedir.

a simple way to do this is to do it in the MDA (maildrop, dovecot-sieve,
... etc). you can do it in postfix, but it requires some work.

> But some people want to receive their spam, so those are
> the exceptions.

It is wrong to ask for the solution of too many problems. one problem at
a time.

>
> So, is this possible with postfix alone, and without spamassassin ?
>

the answer depends on what "this" you mean. you asked multiple
questions... more than one is too much ;-p




Reply | Threaded
Open this post in threaded view
|

Re: Header check and script

Ville Walveranta
I've been looking for the last day (on and off :-) at Todd Bennett's
smtpprox (see http://bent.latency.net/smtpprox/ ). It would likely do
what you're looking to do with minimal modifications.

My goal is to look for the Subject of the arriving messages (all
arriving messages will be filtered through an external, commercial
spam filtering service, emails are not accepted from elsewhere except
for relay by local users and via SASL auth), and those messages whose
Subject begins "**SPAM**" are processed so that 1) the subject line is
rewritten so that the "**SPAM**" text is removed, and 2) an extra
header ("X-Spam: yes") is added that can subsequently be used by Sieve
plugin in Dovecot to deposit the spam into spam mailbox (which is then
pruned by a cron job to last 30 days).

I have the following items added to the configuration:

---[begin master.cf addition]---
#scanner interface
scan       unix  -       -       n       -       10      smtp
   -o smtp_send_xforward_command=yes
   -o disable_mime_output_conversion=yes
   -o smtp_generic_maps=

#return interface for the filtered mailman
localhost:10026 inet  n       -       n       -       10      smtpd
   -o content_filter=
   -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks,no_milters
   -o smtpd_helo_restrictions=
   -o smtpd_client_restrictions=
   -o smtpd_sender_restrictions=
   -o smtpd_recipient_restrictions=permit_mynetworks,reject
   -o mynetworks=127.0.0.0/8
   -o smtpd_authorized_xforward_hosts=127.0.0.0/8
---[end master.cf addition]---

---[begin main.cf addition]---
content_filter = scan:[127.0.0.1]:10025
receive_override_options = no_address_mappings
---[end main.cf addition]---


Todd Bennett's smtpprox forks/kills its own children, and it also
expects a port to bind to, such as with a command

/usr/local/etc/smtpprox/smtpprox localhost:10025 localhost:10026

where the `localhost:10025' is the inbound interface referred to from
Postfix's main.cf, and `localhost:10026' is the interface through
which the processed emails are injected back into the queue. For these
reasons I think it may not be a good idea to use Postfix's spawn to
run smtpprox (especially since the number of concurrent processes set
in master.cf would be multiplied by ten as smtpprox forks up to ten
children), so it's probably best to daemonize it separately.

With the filter set up, the only thing remaining is to modify smtpprox
(which is written in Perl) to do what you want it to do. That's where
I'm at right now; my Perl-skills are very rusty, so I'm trying to
figure out what smtpprox actually does. Looking at Clifton Royston's
header insert patch has helped some, but if someone here is using
smtpprox for a similar purpose (e.g. for mogrifying header or content
in some way), it would be very useful to see your changes to smtpprox.

Ville
Reply | Threaded
Open this post in threaded view
|

Re: Header check and script

mouss-2
Ville Walveranta wrote:
> I've been looking for the last day (on and off :-) at Todd Bennett's
> smtpprox (see http://bent.latency.net/smtpprox/ ). It would likely do
> what you're looking to do with minimal modifications.
>
> My goal is to look for the Subject of the arriving messages

you should have posted this to the other thread:
        "Rewriting Subject line, adding an X-header?"
(always think of the archives).

> (all
> arriving messages will be filtered through an external, commercial
> spam filtering service, emails are not accepted from elsewhere except
> for relay by local users and via SASL auth), and those messages whose
> Subject begins "**SPAM**" are processed so that 1) the subject line is
> rewritten so that the "**SPAM**" text is removed,

note that just because you find **SPAM** in the Subject doesn't mean it
was added by your filtering provider. we see such headers in mail to
mailing-lists (and even in mail from customers;-p)... In short, it does
not prove that the message is spam according to your own policy.

not a serious issue, but be aware of that.


> and 2) an extra
> header ("X-Spam: yes") is added that can subsequently be used by Sieve
> plugin in Dovecot to deposit the spam into spam mailbox (which is then
> pruned by a cron job to last 30 days).
>
> I have the following items added to the configuration:
>
> ---[begin master.cf addition]---
> #scanner interface
> scan       unix  -       -       n       -       10      smtp
>    -o smtp_send_xforward_command=yes
>    -o disable_mime_output_conversion=yes
>    -o smtp_generic_maps=
>
> #return interface for the filtered mailman
> localhost:10026 inet  n       -       n       -       10      smtpd
>    -o content_filter=
>    -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks,no_milters
>    -o smtpd_helo_restrictions=
>    -o smtpd_client_restrictions=
>    -o smtpd_sender_restrictions=
>    -o smtpd_recipient_restrictions=permit_mynetworks,reject
>    -o mynetworks=127.0.0.0/8
>    -o smtpd_authorized_xforward_hosts=127.0.0.0/8
> ---[end master.cf addition]---
>
> ---[begin main.cf addition]---
> content_filter = scan:[127.0.0.1]:10025
> receive_override_options = no_address_mappings
> ---[end main.cf addition]---
>
>
> Todd Bennett's smtpprox forks/kills its own children, and it also
> expects a port to bind to, such as with a command
>
> /usr/local/etc/smtpprox/smtpprox localhost:10025 localhost:10026
>
> where the `localhost:10025' is the inbound interface referred to from
> Postfix's main.cf, and `localhost:10026' is the interface through
> which the processed emails are injected back into the queue. For these
> reasons I think it may not be a good idea to use Postfix's spawn to
> run smtpprox (especially since the number of concurrent processes set
> in master.cf would be multiplied by ten as smtpprox forks up to ten
> children), so it's probably best to daemonize it separately.

yes, don't spawn it.

>
> With the filter set up, the only thing remaining is to modify smtpprox
> (which is written in Perl) to do what you want it to do. That's where
> I'm at right now; my Perl-skills are very rusty, so I'm trying to
> figure out what smtpprox actually does. Looking at Clifton Royston's
> header insert patch has helped some, but if someone here is using
> smtpprox for a similar purpose (e.g. for mogrifying header or content
> in some way), it would be very useful to see your changes to smtpprox.
>


use postfix to prepend the spam header (X-Spam: yes) and use the proxy
to "fix" the subject. This way you prepend the new header instead of
adding it to the middle of previous headers (and you don't want to
buffer headers in the proxy).

to fix the subject, you probably only need to change the yammer()
function, just after
        s/^\./../;

the quick&dirty way (for testing) is to add
        s/^Subject:\s*\*\*SPAM\*\*\s+/Subject: /i;
CAVEAT: this would perform the substitution even in the body. you need a
little more code to prevent this.
Reply | Threaded
Open this post in threaded view
|

Re: Header check and script

Ville Walveranta
On Sun, Jul 27, 2008 at 11:13 AM, mouss <[hidden email]> wrote:
> you should have posted this to the other thread:
>        "Rewriting Subject line, adding an X-header?"
> (always think of the archives).

Indeed. There seems to be a lot of good stuff in the archives!

> note that just because you find **SPAM** in the Subject doesn't mean it was
> added by your filtering provider. we see such headers in mail to
> mailing-lists (and even in mail from customers;-p)... In short, it does not
> prove that the message is spam according to your own policy.

That's true. That's why I wish they would identify spam by other means
than by a simple (and not necessarily unique as you point out) Subject
tagging. An X-Header of some sort would be a better choice. Local spam
filtering is always an option, but takes much more local system
resources and for a relatively small environment subscribing to
commercial RBLs may not be as cost-effective as using an external
commercial filter. For many years I was running
SpamAssassin+dcc-dccd+pyzor combination (with qmail), and it did a
fairly decent job, but in last year or so the amount of spam that was
getting past the filter became unbearable, and the choices became to
either bolster the filtering locally or to externalize it.

> use postfix to prepend the spam header (X-Spam: yes) and use the proxy to
> "fix" the subject. This way you prepend the new header instead of adding it
> to the middle of previous headers (and you don't want to buffer headers in
> the proxy).

That is probably a better way. I was thinking of scanning the message
twice in the proxy; first to look for a Subject line with "**SPAM**";
if found, set a local variable to denote it, then rewrite (add header,
clean the Subject) on the second pass. But that would come at cost of
reduced performance.

> to fix the subject, you probably only need to change the yammer() function,
> just after
>        s/^\./../;
>
> the quick&dirty way (for testing) is to add
>        s/^Subject:\s*\*\*SPAM\*\*\s+/Subject: /i;
> CAVEAT: this would perform the substitution even in the body. you need a
> little more code to prevent this.

Thanks; I'll do some experimentation with the proxy today and that'll
get me started.

Ville