Could you help me with Postfix + MimeDefang?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Could you help me with Postfix + MimeDefang?

Mark Alan-2
Hello list,

I would like to use MimeDefang to sanitize the emails that arrive
at one of our 3 mailing lists, i.e., to convert html->text, remove
unsafe attachments, and remove+webserve file attachments
larger than 500KB.

There are few tutorials on this subject and most, like Mickey Hill's
http://www.mickeyhill.com/mimedefang-howto , ask for the installation
of sendmail and present a config tightly coupled with sendmail
internals (the real sendmail, not postfix's sendmail).

Could you please provide (or point to) a couple of working examples on
how to setup mimedefang with postfix (would it be better done as a
transport+filter, or as milter?) and, if possible, throw some light
on the advantages/disadvantages of each alternative?

Thank you,

Mark
PS: My current setup uses postfix  2.9.3 + mlmmj 1.2.18 + ubuntu
12.04, to manage 2 low volume mailing lists (< 300 subscribers and
less than 50 emails/month), as follows:

MAILING_LIST_1 <[hidden email]>
MAILING_LIST_2 <[hidden email]>

postconf -e 'virtual_alias_maps = /etc/postfix/virtual-alias-maps
# /etc/postfix/virtual-alias-maps
        [hidden email] [hidden email]
        [hidden email] [hidden email]

postconf -e 'transport_maps = /etc/postfix/virtual-transport'
# /etc/postfix/virtual-transport
        [hidden email] mlmmj:mailing_list_1
        [hidden email] mlmmj:mailing_list_2

# /etc/postfix/master.cf
        # transport for the mlmmj mailing list manager
        mlmmj   unix  -       n       n       -       -       pipe
          flags=ORhu user=mlmmj argv=/usr/bin/mlmmj-receive -F
-L /var/spool/mlmmj/$nexthop
Reply | Threaded
Open this post in threaded view
|

Re: Could you help me with Postfix + MimeDefang?

Noel Jones-2
On 2/4/2013 4:14 AM, Mark Alan wrote:

> Hello list,
>
> I would like to use MimeDefang to sanitize the emails that arrive
> at one of our 3 mailing lists, i.e., to convert html->text, remove
> unsafe attachments, and remove+webserve file attachments
> larger than 500KB.
>
> There are few tutorials on this subject and most, like Mickey Hill's
> http://www.mickeyhill.com/mimedefang-howto , ask for the installation
> of sendmail and present a config tightly coupled with sendmail
> internals (the real sendmail, not postfix's sendmail).
>
> Could you please provide (or point to) a couple of working examples on
> how to setup mimedefang with postfix (would it be better done as a
> transport+filter, or as milter?) and, if possible, throw some light
> on the advantages/disadvantages of each alternative?
>
> Thank you,
>
> Mark


mimedefang works as a milter, so that's how you must interface it
with postfix.

All mimedefang configuration is done within mimedefang, not within
the host mailer.  So other than the few milter interface details, it
doesn't really matter if you're using sendmail(TM) or postfix.

Configuring the postfix milter interface is very easy.  Although
there are a bunch of postfix milter options, most of them have
sensible defaults and will not need to be changed.

With a typical milter listening on a local TCP port, all you need is
a postfix main.cf line something like:
# main.cf
smtpd_milters = inet:127.0.0.1:portnum
where portnum matches where the milter application is configured to
listen.  Using a local TCP port is the easiest and simplest way to
configure a milter.

Configuring postfix for a milter listening on a unix socket is similar:
# main.cf
smtpd_milters = unix:/path/to/socket
Figuring out the proper path and permissions can be a little tricky,
especially if postfix is running with the chroot flag set.  Using a
local socket can be a little more secure, but has no performance
benefits.

Full docs are here:
http://www.postfix.org/MILTER_README.html
the config details you will mostly be interested in:
http://www.postfix.org/MILTER_README.html#config



  -- Noel Jones
Reply | Threaded
Open this post in threaded view
|

Re: Could you help me with Postfix + MimeDefang?

Mark Alan-2
On Mon, 04 Feb 2013 09:40:41 -0600, Noel Jones <[hidden email]>
wrote:
> On 2/4/2013 4:14 AM, Mark Alan wrote:

> > I would like to use MimeDefang to sanitize the emails that arrive
> > at one of our 3 mailing lists, i.e., to convert html->text, remove
> > unsafe attachments, and remove+webserve file attachments
> > larger than 500KB.
> >
> > There are few tutorials on this subject and most, like Mickey Hill's
> > http://www.mickeyhill.com/mimedefang-howto , ask for the
> > installation of sendmail and present a config tightly coupled with
> > sendmail internals (the real sendmail, not postfix's sendmail).
> >
> > Could you please provide (or point to) a couple of working examples
> > on how to setup mimedefang with postfix (would it be better done as
> > a transport+filter, or as milter?) and, if possible, throw some
> > light on the advantages/disadvantages of each alternative?
> >
> > Thank you,
> >
> > Mark
>
>
> mimedefang works as a milter, so that's how you must interface it
> with postfix.
> .../...
> the config details you will mostly be interested in:
> http://www.postfix.org/MILTER_README.html#config

First, thank you Noel for sharing your experience and spending your time
at trying to help.

Regarding mimedefang, and its ability to work as a milter, and the
general setup of a milter under postfix, well... I have been there and
done that (in due time, I even described in this list, the config that
we use to run opendkim as a milter, communicating by a unix socket with
a chrooted postfix).

But the question here was entirely different:   "... to use MimeDefang
to sanitize the emails that arrive at ONE of our 3 mailing lists"

The problem was not to apply mimedefang to all incoming mail (like a
milter base config usually does).
The problem is how to do it in order to to process a SINGLE target
email address (the address of a given mailing list), without consuming
unnecessary machine resources, i.e., without "miltering" all the email
that arrives at the postfix server.
That was why I also attached the main.cf/master.cf filter+transport
config that we use to pipe the emails addressed to mailing lists into
the mailing list management software.

Thank you,

Mark
Reply | Threaded
Open this post in threaded view
|

Re: Could you help me with Postfix + MimeDefang?

Noel Jones-2
On 2/4/2013 2:26 PM, Mark Alan wrote:

>
> But the question here was entirely different:   "... to use MimeDefang
> to sanitize the emails that arrive at ONE of our 3 mailing lists"
>
> The problem was not to apply mimedefang to all incoming mail (like a
> milter base config usually does).
> The problem is how to do it in order to to process a SINGLE target
> email address (the address of a given mailing list), without consuming
> unnecessary machine resources, i.e., without "miltering" all the email
> that arrives at the postfix server.

Sorry, missed that part.

A milter applies to all mail.  If you want the milter to only
process some mail, hopefully there are controls within the milter
application for that.




  -- Noel Jones
Reply | Threaded
Open this post in threaded view
|

Re: Could you help me with Postfix + MimeDefang?

Bill Cole-3
In reply to this post by Mark Alan-2
On 4 Feb 2013, at 15:26, Mark Alan wrote:

> The problem is how to do it in order to to process a SINGLE target
> email address (the address of a given mailing list), without consuming
> unnecessary machine resources, i.e., without "miltering" all the email
> that arrives at the postfix server.

As Noel said, Postfix offers no way to do that. You could do it inside
MD, but it won't be very resource-sparing to do so because MD will build
its per-message working directory for you before your filter can tell it
to pass the message along unmolested. An alternative to hooking the MD
milter into your main smtpd would be to define a transport in master.cf
running smtpd with MD as a milter, and use postfix's transport map to
route just the one address there. This would also allow you to avoid the
ugly problem of envelope recipient splitting inside MD. You can probably
get a more complete answer on the MD mailing list. Also note that
configuring MD means writing a collection of Perl functions with
predefined interfaces to implement the message filtering. If you are not
comfortable writing Perl, MD may not be the right tool for you.
Reply | Threaded
Open this post in threaded view
|

Re: Could you help me with Postfix + MimeDefang?

Noel Jones-2
On 2/4/2013 9:13 PM, Bill Cole wrote:

> On 4 Feb 2013, at 15:26, Mark Alan wrote:
>
>> The problem is how to do it in order to to process a SINGLE target
>> email address (the address of a given mailing list), without
>> consuming
>> unnecessary machine resources, i.e., without "miltering" all the
>> email
>> that arrives at the postfix server.
>
> As Noel said, Postfix offers no way to do that. You could do it
> inside MD, but it won't be very resource-sparing to do so because MD
> will build its per-message working directory for you before your
> filter can tell it to pass the message along unmolested. An
> alternative to hooking the MD milter into your main smtpd would be
> to define a transport in master.cf running smtpd with MD as a
> milter, and use postfix's transport map to route just the one
> address there. This would also allow you to avoid the ugly problem
> of envelope recipient splitting inside MD. You can probably get a
> more complete answer on the MD mailing list. Also note that
> configuring MD means writing a collection of Perl functions with
> predefined interfaces to implement the message filtering. If you are
> not comfortable writing Perl, MD may not be the right tool for you.

Good suggestion, Bill.

If mimedefang is intended to only "mangle" messages, a per-user
transport into a different postfix instance would work fine.

If mimedefang might sometimes reject a message, a transport would
not be acceptable as it would cause backscatter by bouncing the
message to a possibly forged sender address.



  -- Noel Jones
Reply | Threaded
Open this post in threaded view
|

Re: Could you help me with Postfix + MimeDefang?

Mark Alan-2
In reply to this post by Bill Cole-3
On Mon, 04 Feb 2013 22:13:14 -0500, "Bill Cole"
<[hidden email]> wrote:

> alternative to hooking the MD milter into your main smtpd would be to
> define a transport in master.cf running smtpd with MD as a milter,
> and use postfix's transport map to route just the one address there.
> This would also allow you to avoid the ugly problem of envelope
> recipient splitting inside MD.

Well, that was also my gut feeling, that was why I posted here to try
to find some (solid) evidence.

So, assuming MD SPOOLDIR='/var/spool/postfix/mimedefang' and
SOCKET='/var/spool/postfix/mimedefang/mimedefang.sock', would the
following do the job?

postconf -e 'virtual_alias_maps = /etc/postfix/virtual-alias-maps
# /etc/postfix/virtual-alias-maps
   [hidden email] [hidden email]
   ...
postconf -e 'transport_maps = /etc/postfix/virtual-transport'
# /etc/postfix/virtual-transport
   [hidden email] filteredmlmmj:mailing_list_1
   ...
# /etc/postfix/master.cf
   # transport for the mlmmj mailing lists
    mlmmj   unix  -       n       n       -       -       pipe
      flags=ORhu user=mlmmj argv=/usr/bin/mlmmj-receive -F
-L /var/spool/mlmmj/$nexthop
   # filtered transport for the mlmmj mailing list manager
    filteredmlmmj   unix  -       -       -       -       - mlmmj
      -o smtpd_milters = unix:mimedefang/mimedefang.sock

Please note, in this last statement, 'unix', 'mlmmj' and '-o
smtpd_milters' nad the 5 dashes.

> You can probably get a more complete answer on the MD mailing list.

Not at all. The stated problem is an old problem. I have researched
extensively a lot of discussions about this subject both in MD list and
in the postfix list (and a lot of useless 'recipes' too). People tend
to see this issue as some sort of magically solved hit or miss issue.
And the people that develop MD seems to be more in the business of
selling "canned" solutions (pun intended), than into producing good
and clear documentation.

> Also note that configuring MD means
> writing a collection of Perl functions with predefined interfaces to
> implement the message filtering. If you are not comfortable writing
> Perl,

No problem with the needed Perl functions.

> MD may not be the right tool for you.

MD is certainly resources hungry. But I do not know any other app
that meets the specs:
  convert html->text,
  remove unsafe attachments (offenders with known ext's),
  remove+webserve file attachments larger than 500KB

Right now we are piping email into "altermime --input=- --removeall",
but altermime is orphaned/abandonware and it does not do that file
attachment remove+webserve job.

Thank you,

Mark.
Reply | Threaded
Open this post in threaded view
|

Re: Could you help me with Postfix + MimeDefang?

Bill Cole-3
On 5 Feb 2013, at 4:46, Mark Alan wrote:

> On Mon, 04 Feb 2013 22:13:14 -0500, "Bill Cole"
> <[hidden email]> wrote:
>
>> alternative to hooking the MD milter into your main smtpd would be to
>> define a transport in master.cf running smtpd with MD as a milter,
>> and use postfix's transport map to route just the one address there.
>> This would also allow you to avoid the ugly problem of envelope
>> recipient splitting inside MD.
>
> Well, that was also my gut feeling, that was why I posted here to try
> to find some (solid) evidence.
>
> So, assuming MD SPOOLDIR='/var/spool/postfix/mimedefang' and
> SOCKET='/var/spool/postfix/mimedefang/mimedefang.sock',

Bad idea. Don't put non-postfix stuff in /var/spool/postfix/. The
default
SPOOLDIR='/var/spool/MIMEDefang' and
SOCKET='/var/spool/MIMEDefang/mimedefang.sock' are fine.

> would the
> following do the job?
>
> postconf -e 'virtual_alias_maps = /etc/postfix/virtual-alias-maps
> # /etc/postfix/virtual-alias-maps
> [hidden email] [hidden email]
> ...
> postconf -e 'transport_maps = /etc/postfix/virtual-transport'
> # /etc/postfix/virtual-transport
> [hidden email] filteredmlmmj:mailing_list_1
> ...
> # /etc/postfix/master.cf
> # transport for the mlmmj mailing lists
> mlmmj   unix  -       n       n       -       -       pipe
> flags=ORhu user=mlmmj argv=/usr/bin/mlmmj-receive -F
> -L /var/spool/mlmmj/$nexthop
> # filtered transport for the mlmmj mailing list manager
> filteredmlmmj   unix  -       -       -       -       - mlmmj
> -o smtpd_milters = unix:mimedefang/mimedefang.sock
>
> Please note, in this last statement, 'unix', 'mlmmj' and '-o
> smtpd_milters' nad the 5 dashes.

Substantially wrong. You would need to define a new transport in
master.cf running *smtpd* in the manner used for a submission daemon or
the transport used for amavisd output. e.g. something like this:

localhost:10025 inet n - n - - smtpd
    -o smtpd_milters=unix:/var/spool/MIMEDefang/mimedefang.sock
    -o transport_maps=
(maybe other -o lines to override main.cf)

I don't use MD or Postfix that way myself, so I'm a bit leery of cooking
up anything that looks like a complete example. The concept is that you
are routing some mail through another postfix SMTP daemon instance with
its own independent configuration.

>> You can probably get a more complete answer on the MD mailing list.
>
> Not at all. The stated problem is an old problem. I have researched
> extensively a lot of discussions about this subject both in MD list
> and
> in the postfix list (and a lot of useless 'recipes' too). People tend
> to see this issue as some sort of magically solved hit or miss issue.

Or irrelevant to them. What you want to do isn't something most people
using MD or Postfix want to do, and most people using one or the other
are not using them together. It seems to me that the best way to do this
would be to let MD see everything rather than to try to route around it,
but in any case most of your work will be in writing the filter
routines, not doing the plumbing. The experts in that aspect will be on
the MD list more than they will be here.

> And the people that develop MD seems to be more in the business of
> selling "canned" solutions (pun intended), than into producing good
> and clear documentation.

There used to be a wiki, but it got overrun with spam. The docs in the
distribution are pretty solid and the code is not hard to follow. The
LISA '04 presentation on the website is actually quite good at providing
a high-level view. If you're looking for 'recipes' that will work for
your particular need you aren't going to find them because that's just
not the sort of tool MD is. It is written for Perl coders who also run
mail systems, not mail admins who can read Perl. Manipulating mail
content has become increasingly unpopular over the past decade for
mostly good reasons and it's relatively rare to use MD solely for that
purpose or to only deploy it for some addresses. It's not remarkable
that no one has a setup for addressing your issue that they feel
comfortable sharing. I've been using MD for a decade with both Sendmail
and Postfix and only have a general idea of how to approach it.


>> Also note that configuring MD means
>> writing a collection of Perl functions with predefined interfaces to
>> implement the message filtering. If you are not comfortable writing
>> Perl,
>
> No problem with the needed Perl functions.
>
>> MD may not be the right tool for you.
>
> MD is certainly resources hungry. But I do not know any other app
> that meets the specs:
> convert html->text,
> remove unsafe attachments (offenders with known ext's),
> remove+webserve file attachments larger than 500KB
>
> Right now we are piping email into "altermime --input=- --removeall",
> but altermime is orphaned/abandonware and it does not do that file
> attachment remove+webserve job.

You might want to consider just writing your own pipe-suitable delivery
filter using the MIME-Tools suite rather than loading up all the
superstructure of MD. It seems like you would be using a small subset of
MD itself and coding most of what you actually want done in your filter
and filter_multipart functions, so why bother with MD?

Reply | Threaded
Open this post in threaded view
|

Re: Could you help me with Postfix + MimeDefang?

James Griffin
In reply to this post by Noel Jones-2
--> Noel Jones <[hidden email]> [2013-02-04 14:56:23 -0600]:

> On 2/4/2013 2:26 PM, Mark Alan wrote:
>
> >
> > But the question here was entirely different:   "... to use MimeDefang
> > to sanitize the emails that arrive at ONE of our 3 mailing lists"
> >
> > The problem was not to apply mimedefang to all incoming mail (like a
> > milter base config usually does).
> > The problem is how to do it in order to to process a SINGLE target
> > email address (the address of a given mailing list), without consuming
> > unnecessary machine resources, i.e., without "miltering" all the email
> > that arrives at the postfix server.
>
> Sorry, missed that part.
>
> A milter applies to all mail.  If you want the milter to only
> process some mail, hopefully there are controls within the milter
> application for that.

I believe demime would/could achieve what you're looking for. It
is used on a few mailing lists i'm subscribed to and it's quite
easy to set up. Have you considered that or other alternatives?


--
Primary Key: 4096R/1D31DC38 2011-12-03
Key Fingerprint: A4B9 E875 A18C 6E11 F46D  B788 BEE6 1251 1D31 DC38
Reply | Threaded
Open this post in threaded view
|

Re: Could you help me with Postfix + MimeDefang?

Noel Jones-2
In reply to this post by Bill Cole-3
On 2/5/2013 10:47 AM, Bill Cole wrote:

> On 5 Feb 2013, at 4:46, Mark Alan wrote:
>
>> On Mon, 04 Feb 2013 22:13:14 -0500, "Bill Cole"
>> <[hidden email]> wrote:
>>
>>> alternative to hooking the MD milter into your main smtpd would
>>> be to
>>> define a transport in master.cf running smtpd with MD as a milter,
>>> and use postfix's transport map to route just the one address there.
>>> This would also allow you to avoid the ugly problem of envelope
>>> recipient splitting inside MD.
>>
>> Well, that was also my gut feeling, that was why I posted here to try
>> to find some (solid) evidence.
>>
>> So, assuming MD SPOOLDIR='/var/spool/postfix/mimedefang' and
>> SOCKET='/var/spool/postfix/mimedefang/mimedefang.sock',
>
> Bad idea. Don't put non-postfix stuff in /var/spool/postfix/. The
> default
> SPOOLDIR='/var/spool/MIMEDefang' and
> SOCKET='/var/spool/MIMEDefang/mimedefang.sock' are fine.
>
>> would the
>> following do the job?
>>
>> postconf -e 'virtual_alias_maps = /etc/postfix/virtual-alias-maps
>> # /etc/postfix/virtual-alias-maps
>> [hidden email] [hidden email]
>> ...
>> postconf -e 'transport_maps = /etc/postfix/virtual-transport'
>> # /etc/postfix/virtual-transport
>> [hidden email] filteredmlmmj:mailing_list_1
>> ...
>> # /etc/postfix/master.cf
>> # transport for the mlmmj mailing lists
>> mlmmj   unix  -       n       n       -       -       pipe
>> flags=ORhu user=mlmmj argv=/usr/bin/mlmmj-receive -F
>> -L /var/spool/mlmmj/$nexthop
>> # filtered transport for the mlmmj mailing list manager
>> filteredmlmmj   unix  -       -       -       -       - mlmmj
>> -o smtpd_milters = unix:mimedefang/mimedefang.sock
>>
>> Please note, in this last statement, 'unix', 'mlmmj' and '-o
>> smtpd_milters' nad the 5 dashes.
>
> Substantially wrong. You would need to define a new transport in
> master.cf running *smtpd* in the manner used for a submission daemon
> or the transport used for amavisd output. e.g. something like this:
>
> localhost:10025 inet n - n - - smtpd
>    -o smtpd_milters=unix:/var/spool/MIMEDefang/mimedefang.sock
>    -o transport_maps=
> (maybe other -o lines to override main.cf)


You're on the right track here.

The transport table cannot be overridden with a -o option, so this
would need to go through a separate postfix instance, not just a
different smtpd listener.
http://www.postfix.org/MULTI_INSTANCE_README.html


On 2/5/2013 11:23 AM, James Griffin wrote:
> I believe demime would/could achieve what you're looking for. It
> is used on a few mailing lists i'm subscribed to and it's quite
> easy to set up. Have you considered that or other alternatives?

That sounds like a much easier solution.





  -- Noel Jones