regexp help for header_checks

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

regexp help for header_checks

James M
The distro example provided does not work for me. I added a couple of extensions as follows..
/^content-(type|disposition):.*name[[:space:]]*=.*\.(exe|vbs|lnk|asd|hlp|ocx|reg|bat|cmd|dll|vxd|pif|scr|hta|sh[mbs]|vb[esx]|ws[fh]|wmf)/ REJECT Suspicious attachment detected
 
The problem is that it matches a legit file such as test.batch.zip. The regexp fails, in my humble opinion, because it fails to account for an attachment with multiple dots as in the preceding example.
I need something that will only test for the last part of the string.
I tried adding a $ or \z to the regexp....i.e.-->
/^content-(type|disposition):.*name[[:space:]]*=.*\.(exe|vbs|lnk|asd|hlp|ocx|reg
|bat|cmd|dll|vxd|pif|scr|hta|sh[mbs]|vb[esx]|ws[fh]|wmf)$/ REJECT Suspicious atta
chment detected
...to no avail.
I notice that sometimes the filename/name is enclosed with "" in the header so that may be why my solution is failing. (lotus notes encloses w/"" and google does not)
Not being a regexp guru I thought I would see if someone on this list may have a solution.
I also tried =[\.|.]*
 
I also don't understand what [[:space:]] means so if you could explain that also it would be a bonus.
Thanks
-J
Reply | Threaded
Open this post in threaded view
|

Re: regexp help for header_checks

Jorey Bump
James M wrote, at 06/13/2008 10:03 AM:
> The distro example provided does not work for me. I added a couple of
> extensions as follows..
> /^content-(type|disposition):.*name[[:space:]]*=.*\.(exe|vbs|lnk|asd|hlp|ocx|reg|bat|cmd|dll|vxd|pif|scr|hta|sh[mbs]|vb[esx]|ws[fh]|wmf)/
> REJECT Suspicious attachment detected
>  
> The problem is that it matches a legit file such as test.batch.zip. The
> regexp fails, in my humble opinion, because it fails to account for an
> attachment with multiple dots as in the preceding example.

No, the expression is matching the ".bat" part of the string, as it is
configured to do. You just need a new expression.

> I need something that will only test for the last part of the string.
> I tried adding a $ or \z to the regexp....i.e.-->
> /^content-(type|disposition):.*name[[:space:]]*=.*\.(exe|vbs|lnk|asd|hlp|ocx|reg
> |bat|cmd|dll|vxd|pif|scr|hta|sh[mbs]|vb[esx]|ws[fh]|wmf)$/ REJECT
> Suspicious atta
> chment detected
> ...to no avail.
> I notice that sometimes the filename/name is enclosed with "" in the
> header so that may be why my solution is failing. (lotus notes encloses
> w/"" and google does not)
> Not being a regexp guru I thought I would see if someone on this list
> may have a solution.
> I also tried =[\.|.]*

I use this (in a pcre header_checks map):

/^Content-(Disposition|Type):\s+.*?(file)?name=.*\.(bat|chm|dll|exe|hta|lnk|pif|scr|vbs)["';]*$/
         REJECT Suspicious attachment

I'm sure there are many variations (arguably, it should also look for
padded spaces before the end of the line, and I can't seem to justify
the sequence of '+.*?'), but I prefer not to go too far overboard for
rules like this.

> I also don't understand what [[:space:]] means so if you could explain
> that also it would be a bonus.

It's a named character class that matches characters recognized as
spaces. \s is typically equivalent, but [[:space:]] may be more
portable, in certain cases. Unless you're affected by the limitations of
your environment, it's a matter of taste.


Reply | Threaded
Open this post in threaded view
|

Re: regexp help for header_checks

Victor Duchovni
On Fri, Jun 13, 2008 at 10:53:03AM -0400, Jorey Bump wrote:

> James M wrote, at 06/13/2008 10:03 AM:
> >The distro example provided does not work for me. I added a couple of
> >extensions as follows..
> >/^content-(type|disposition):.*name[[:space:]]*=.*\.(exe|vbs|lnk|asd|hlp|ocx|reg|bat|cmd|dll|vxd|pif|scr|hta|sh[mbs]|vb[esx]|ws[fh]|wmf)/
> >REJECT Suspicious attachment detected
> >
> >The problem is that it matches a legit file such as test.batch.zip. The
> >regexp fails, in my humble opinion, because it fails to account for an
> >attachment with multiple dots as in the preceding example.
>
> No, the expression is matching the ".bat" part of the string, as it is
> configured to do. You just need a new expression.
>
> >I need something that will only test for the last part of the string.
> >I tried adding a $ or \z to the regexp....i.e.-->
> >/^content-(type|disposition):.*name[[:space:]]*=.*\.(exe|vbs|lnk|asd|hlp|ocx|reg
> >|bat|cmd|dll|vxd|pif|scr|hta|sh[mbs]|vb[esx]|ws[fh]|wmf)$/ REJECT
> >Suspicious atta
> >chment detected
> >...to no avail.
> >I notice that sometimes the filename/name is enclosed with "" in the
> >header so that may be why my solution is failing. (lotus notes encloses
> >w/"" and google does not)
> >Not being a regexp guru I thought I would see if someone on this list
> >may have a solution.
> >I also tried =[\.|.]*
>
> I use this (in a pcre header_checks map):
>
> /^Content-(Disposition|Type):\s+.*?(file)?name=.*\.(bat|chm|dll|exe|hta|lnk|pif|scr|vbs)["';]*$/
>         REJECT Suspicious attachment
>

The file name could be followed by whitespace.

Don't include <'> it is not a special character in MIME. Drop the "$",
there can easily be additional MIME attributes after the file name.

    Content-Disposition: attachment; filename = foo.exe ; x-noise = random

Regexps are not a good way to parse MIME headers. One can at best do a sloppy
job. Robust attachment name processing requires a real MIME parser.

    De-facto standard abuse of RFC 2047:

        Content-Disposition: attachment;
            filename = "=?us-ascii?Q?foo=2eexe?=" ; x-noise = random

    Standard, but not ubiquitous RFC 2231:

        Content-Disposition: attachment;
            filename* = us-ascii'en-us'foo%2eexe ; x-noise = random

    ...

--
        Viktor.

Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.

To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:[hidden email]?body=unsubscribe%20postfix-users>

If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.
Reply | Threaded
Open this post in threaded view
|

Re: regexp help for header_checks

James M



The file name could be followed by whitespace.

Don't include <'> it is not a special character in MIME. Drop the "$",
there can easily be additional MIME attributes after the file name.
 
would wmf)[\s";]/ be appropriate until a better solution is found?

   Content-Disposition: attachment; filename = foo.exe ; x-noise = random

Regexps are not a good way to parse MIME headers. One can at best do a sloppy
job. Robust attachment name processing requires a real MIME parser.

where is this "real MIME parser"? is it part of postfix or an add-on product?
any pointers howto's appreciated.
-J
Reply | Threaded
Open this post in threaded view
|

Re: regexp help for header_checks

Victor Duchovni
On Fri, Jun 13, 2008 at 11:42:22AM -0400, James M wrote:

> > The file name could be followed by whitespace.
> >
> > Don't include <'> it is not a special character in MIME. Drop the "$",
> > there can easily be additional MIME attributes after the file name.
> >
>
> would wmf)[\s";]/ be appropriate until a better solution is found?

This is reasonable.

> > Robust attachment name processing requires a real MIME parser.
>
> where is this "real MIME parser"? is it part of postfix or an add-on
> product?

I use my own, so I don't know what else is out there, but to reject
bad MIME during SMTP you need a SMTP proxy filter that parses MIME
and rejects unwanted file names. Don't know who provides a suitable
parser.

--
        Viktor.

Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.

To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:[hidden email]?body=unsubscribe%20postfix-users>

If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.
Reply | Threaded
Open this post in threaded view
|

Re: regexp help for header_checks

Jorey Bump
In reply to this post by James M
James M wrote, at 06/13/2008 11:42 AM:

>     The file name could be followed by whitespace.
>
>     Don't include <'> it is not a special character in MIME. Drop the "$",
>     there can easily be additional MIME attributes after the file name.
>      
>
> would wmf)[\s";]/ be appropriate until a better solution is found?

You can test expressions on your mailstore with egrep before putting
them into production, which might help to avoid false positives
(obviously, it will only work on existing mail).

  egrep -ri '^Content-(Disposition|Type):[[:space:]]+.*?(file)?name='
/path/to/mailbox

See what this reveals, then use your more narrow expression and compare.
Remember, you're not only trying to catch bad extensions, you are also
trying to minimize or eliminate false positives (which is why ".com" is
in neither of our expressions, even though it is as potentially harmful
as any other).

Reply | Threaded
Open this post in threaded view
|

Re: regexp help for header_checks

Wietse Venema
In reply to this post by Victor Duchovni
Victor Duchovni:
> > > Robust attachment name processing requires a real MIME parser.
> >
> > where is this "real MIME parser"? is it part of postfix or an add-on
> > product?
>
> I use my own, so I don't know what else is out there, but to reject
> bad MIME during SMTP you need a SMTP proxy filter that parses MIME
> and rejects unwanted file names. Don't know who provides a suitable
> parser.

It's worse. Robust attachment scrubbing requires a MIME normalizer
(like Victor uses), because different systems interpret non-compliant
MIME messages differently.

Postfix is not a MIME normalizer.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: regexp help for header_checks

Noel Jones-2
In reply to this post by Jorey Bump
Jorey Bump wrote:

> James M wrote, at 06/13/2008 11:42 AM:
>
>>     The file name could be followed by whitespace.
>>
>>     Don't include <'> it is not a special character in MIME. Drop the
>> "$",
>>     there can easily be additional MIME attributes after the file name.
>>      
>> would wmf)[\s";]/ be appropriate until a better solution is found?
>
> You can test expressions on your mailstore with egrep before putting
> them into production, which might help to avoid false positives
> (obviously, it will only work on existing mail).
>
>  egrep -ri '^Content-(Disposition|Type):[[:space:]]+.*?(file)?name='
> /path/to/mailbox
>
> See what this reveals, then use your more narrow expression and compare.
> Remember, you're not only trying to catch bad extensions, you are also
> trying to minimize or eliminate false positives (which is why ".com" is
> in neither of our expressions, even though it is as potentially harmful
> as any other).
>

There's a very complete and effective pcre expression on the
header_checks man page, in the EXAMPLES section near the end.
Adjust the list of bad extensions to suit your taste, but the
general structure should catch "most" mime variations.
(Deliberate, severe obfuscation might still slip through.)
http://www.postfix.org/header_checks.5.html


--
Noel Jones
Reply | Threaded
Open this post in threaded view
|

Re: regexp help for header_checks

Victor Duchovni
On Fri, Jun 13, 2008 at 11:48:27AM -0500, Noel Jones wrote:

> (Deliberate, severe obfuscation might still slip through.)
> http://www.postfix.org/header_checks.5.html

Replace "might" with "will", but otherwise, the documented patterns are
about as good as you'll get with regular expressions.

--
        Viktor.

Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.

To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:[hidden email]?body=unsubscribe%20postfix-users>

If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.