how to bypass milters, whitelist hosts

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

how to bypass milters, whitelist hosts

martin f krafft-2
Hi,

how can I bypass smtpd_milters for certain hosts?

I have asked a related question previously [0], and the only
solution seemed to be to redirect those hosts to a different smtpd
instance, but unfortunately, Linux cannot redirect IPv6 connections
yet (TPROXY is in preparation).

0. http://www.irbs.net/internet/postfix/0708/0450.html

Is there another way to approach this?

Why are header_checks/body_checks/smtpd_milters not regular
restrictions, which I could put into smtpd_end_of_data_restrictions
after a whitelist?

Thanks,

--
martin;              (greetings from the heart of the sun.)
  \____ echo mailto: !#^."<*>"|tr "<*> mailto:" net@madduck
 
it is ok to let your mind go blank, but please turn off the sound.
 
spamtraps: [hidden email]

digital_signature_gpg.asc (204 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: how to bypass milters, whitelist hosts

Wietse Venema
martin f krafft:
> Hi,
>
> how can I bypass smtpd_milters for certain hosts?

Not.  This question is related to the following question: how
can I change the Milter depending on the client host.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: how to bypass milters, whitelist hosts

martin f krafft-2
also sprach Wietse Venema <[hidden email]> [2009.05.22.1826 +0200]:
> > how can I bypass smtpd_milters for certain hosts?
>
> Not.  This question is related to the following question: how
> can I change the Milter depending on the client host.

Right, but I cannot really find anything on that either.

Why are *_checks and *_milters not end-of-data restrictions, or
better yet, policy services?

--
martin | http://madduck.net/ | http://two.sentenc.es/
 
"we should have a volleyballocracy.
 we elect a six-pack of presidents.
 each one serves until they screw up,
 at which point they rotate."
                                                      -- dennis miller
 
spamtraps: [hidden email]

digital_signature_gpg.asc (204 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: how to bypass milters, whitelist hosts

Sahil Tandon
On Fri, 22 May 2009, martin f krafft wrote:

> also sprach Wietse Venema <[hidden email]> [2009.05.22.1826 +0200]:
> > > how can I bypass smtpd_milters for certain hosts?
> >
> > Not.  This question is related to the following question: how
> > can I change the Milter depending on the client host.
>
> Why are *_checks and *_milters not end-of-data restrictions, or
> better yet, policy services?

One example: 1.2.3.4 is rejected in an access(5) table referenced in
smtpd_client_restrictions.  Why wait for END-OF-DATA when you know, in
advance, that you will not accept mail from 1.2.3.4?

--
Sahil Tandon <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: how to bypass milters, whitelist hosts

martin f krafft-2
also sprach Sahil Tandon <[hidden email]> [2009.05.23.0037 +0200]:
> > Why are *_checks and *_milters not end-of-data restrictions, or
> > better yet, policy services?
>
> One example: 1.2.3.4 is rejected in an access(5) table referenced
> in smtpd_client_restrictions.  Why wait for END-OF-DATA when you
> know, in advance, that you will not accept mail from 1.2.3.4?

I don't see the relation. If milters and content checks were
end-of-data restrictions of policy services, there is nothing
stopping postfix from rejecting a mail at RCPT-time if an earlier
restriction class return failure.

--
martin | http://madduck.net/ | http://two.sentenc.es/
 
military justice is to justice what military music is to music.
                                                       -- groucho marx
 
spamtraps: [hidden email]

digital_signature_gpg.asc (204 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: how to bypass milters, whitelist hosts

fakessh @
martin f krafft a écrit :

> also sprach Sahil Tandon <[hidden email]> [2009.05.23.0037 +0200]:
>>> Why are *_checks and *_milters not end-of-data restrictions, or
>>> better yet, policy services?
>> One example: 1.2.3.4 is rejected in an access(5) table referenced
>> in smtpd_client_restrictions.  Why wait for END-OF-DATA when you
>> know, in advance, that you will not accept mail from 1.2.3.4?
>
> I don't see the relation. If milters and content checks were
> end-of-data restrictions of policy services, there is nothing
> stopping postfix from rejecting a mail at RCPT-time if an earlier
> restriction class return failure.
>

  HI All,
  look maybe amavisd
  check_policy_service are not allowed ( not work fine ) in my box

Reply | Threaded
Open this post in threaded view
|

Re: how to bypass milters, whitelist hosts

Wietse Venema
In reply to this post by martin f krafft-2
martin f krafft:

> also sprach Sahil Tandon <[hidden email]> [2009.05.23.0037 +0200]:
> > > Why are *_checks and *_milters not end-of-data restrictions, or
> > > better yet, policy services?
> >
> > One example: 1.2.3.4 is rejected in an access(5) table referenced
> > in smtpd_client_restrictions.  Why wait for END-OF-DATA when you
> > know, in advance, that you will not accept mail from 1.2.3.4?
>
> I don't see the relation. If milters and content checks were
> end-of-data restrictions of policy services, there is nothing
> stopping postfix from rejecting a mail at RCPT-time if an earlier
> restriction class return failure.

Before making architectural recommendations, it would help to step
back into the reality of how policy servers and milters work.  For
one thing, policy servers don't handle message content, and for
another, Milters must be able to see every SMTP protocol step.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: how to bypass milters, whitelist hosts

Robert Schetterer
Wietse Venema schrieb:

> martin f krafft:
>> also sprach Sahil Tandon <[hidden email]> [2009.05.23.0037 +0200]:
>>>> Why are *_checks and *_milters not end-of-data restrictions, or
>>>> better yet, policy services?
>>> One example: 1.2.3.4 is rejected in an access(5) table referenced
>>> in smtpd_client_restrictions.  Why wait for END-OF-DATA when you
>>> know, in advance, that you will not accept mail from 1.2.3.4?
>> I don't see the relation. If milters and content checks were
>> end-of-data restrictions of policy services, there is nothing
>> stopping postfix from rejecting a mail at RCPT-time if an earlier
>> restriction class return failure.
>
> Before making architectural recommendations, it would help to step
> back into the reality of how policy servers and milters work.  For
> one thing, policy servers don't handle message content, and for
> another, Milters must be able to see every SMTP protocol step.
>
> Wietse

Hi Martin, after all most milters have option to whitelist hosts itself
why dont use it

--
Best Regards

MfG Robert Schetterer

Germany/Munich/Bavaria
Reply | Threaded
Open this post in threaded view
|

Re: how to bypass milters, whitelist hosts

Kouhei Sutou-2
In reply to this post by martin f krafft-2
Hi,

In <[hidden email]>
  "how to bypass milters, whitelist hosts" on Fri, 22 May 2009 15:51:10 +0200,
  martin f krafft <[hidden email]> wrote:

> how can I bypass smtpd_milters for certain hosts?
>
> I have asked a related question previously [0], and the only
> solution seemed to be to redirect those hosts to a different smtpd
> instance, but unfortunately, Linux cannot redirect IPv6 connections
> yet (TPROXY is in preparation).
>
> 0. http://www.irbs.net/internet/postfix/0708/0450.html
>
> Is there another way to approach this?

You can do it with milter manager:
  http://milter-manager.sourceforge.net/

milter manager is placed at between Postfix and milters:

  Postfix <-milter protocol-> milter manager <-milter protocol-> milters

milter manager can bypass your milter if connected host is
whitelisted host.


If you want more information, I'll describe more details
about milter manager.


Thanks,
--
kou
Reply | Threaded
Open this post in threaded view
|

Re: how to bypass milters, whitelist hosts

martin f krafft-2
In reply to this post by Robert Schetterer
also sprach Robert Schetterer <[hidden email]> [2009.05.23.2244 +0200]:
> Hi Martin, after all most milters have option to whitelist hosts
> itself why dont use it

Because it means I have to maintain redunant list of exempt hosts.

--
martin | http://madduck.net/ | http://two.sentenc.es/
 
"die zeit für kleine politik ist vorbei.
 schon das nächste jahrhundert
 bringt den kampf um die erdherrschaft."
                                                 - friedrich nietzsche
 
spamtraps: [hidden email]

digital_signature_gpg.asc (204 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: how to bypass milters, whitelist hosts

martin f krafft-2
In reply to this post by Wietse Venema
also sprach Wietse Venema <[hidden email]> [2009.05.23.1442 +0200]:
> Before making architectural recommendations, it would help to step
> back into the reality of how policy servers and milters work.  For
> one thing, policy servers don't handle message content, and for
> another, Milters must be able to see every SMTP protocol step.

I didn't know that. Milters are then co-processes and smtpd proxies
the client commands to them, receives the milter decisions, and
reacts accordingly.

I assume that milters have the concept of an SMTP transactions,
meaning they would not like it a whole lot if RCPT was proxied
before EHLO and MAIL.

Then, there seem to be only two ways to whitelist hosts:

1. decide at connect() time whether the milter should be consulted
   for this transaction at all, or maybe decide /which/ milters
   should run. I imagine this could be done with a hash/cidr_table
   lookup on the client IP, where the hash value is the set of
   milters to run. $smtpd_milters would then take a list, or
   a table, and an existing setup of $smtpd_milters=foo,bar could be
   migrated by using e.g. a cidr_table entry like

     0.0.0.0/0 foo,bar

2. Each smtpd_*_restrictions class could get an entry similar to
   map_milter_decision, which is an identity map by default, but
   could be overridden to make e.g. a REJECT into a DISCARD at this
   stage of the SMTP transaction. Connections from whitelisted
   clients could be routed to an overriding map of this kind with
   smtpd_restriction_classes.

Are there other problems with these approaches I am currently not
seeing?

--
martin | http://madduck.net/ | http://two.sentenc.es/
 
"women, when they are not in love,
 have all the cold blood of an experienced attorney."
                                                   -- honoré de balzac
 
spamtraps: [hidden email]

digital_signature_gpg.asc (204 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

header/body_checks as end-of-data checks (was: how to bypass milters, whitelist hosts)

martin f krafft-2
In reply to this post by Wietse Venema
also sprach Wietse Venema <[hidden email]> [2009.05.23.1442 +0200]:
> Before making architectural recommendations, it would help to step
> back into the reality of how policy servers and milters work.  For
> one thing, policy servers don't handle message content, and for
> another, Milters must be able to see every SMTP protocol step.

What about header/body_checks? Those would make sense in
smtpd_end_of_data_restrictions, wouldn't they? What am I missing to
understand why they aren't? Is it the fact that those checks should
apply to all mail processed by postfix, whether received by smtpd or
not?

--
martin | http://madduck.net/ | http://two.sentenc.es/
 
"syntactic sugar causes cancer of the semicolon."
                                            -- epigrams in programming
 
spamtraps: [hidden email]

digital_signature_gpg.asc (204 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: how to bypass milters, whitelist hosts

martin f krafft-2
In reply to this post by Kouhei Sutou-2
also sprach Kouhei Sutou <[hidden email]> [2009.05.25.0148 +0200]:
> milter manager is placed at between Postfix and milters:
>
>   Postfix <-milter protocol-> milter manager <-milter protocol->
>   milters
>
> milter manager can bypass your milter if connected host is
> whitelisted host.

While this is definitely helpful, it does mean that I have to
maintain the list of exempted hosts outside inside and outside of
postfix, which is redundancy I'd rather avoid.

Thanks for the pointer though, I'll check it out.

--
martin | http://madduck.net/ | http://two.sentenc.es/
 
murphy's law is recursive.
washing your car to make it rain doesn't work.
 
spamtraps: [hidden email]

digital_signature_gpg.asc (204 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: how to bypass milters, whitelist hosts

Kouhei Sutou-2
Hi,

In <[hidden email]>
  "Re: how to bypass milters, whitelist hosts" on Mon, 25 May 2009 11:51:36 +0200,
  martin f krafft <[hidden email]> wrote:

> also sprach Kouhei Sutou <[hidden email]> [2009.05.25.0148 +0200]:
>> milter manager is placed at between Postfix and milters:
>>
>>   Postfix <-milter protocol-> milter manager <-milter protocol->
>>   milters
>>
>> milter manager can bypass your milter if connected host is
>> whitelisted host.
>
> While this is definitely helpful, it does mean that I have to
> maintain the list of exempted hosts outside inside and outside of
> postfix, which is redundancy I'd rather avoid.

What format are you using for whitelist?

If you are using simple access(5) format, you can reuse it
easily because milter manager has embedded Ruby interpreter.
access(5) format can be parsed easily with Ruby.

milter-manager.conf:
  ...

  # Parse simple access(5) format:
  #   good.example.com OK
  #   bad.example.com REJECT
  whitelist = {}
  File.read("/.../whitelist").each_line do |line|
    next if /^#/ =~ line
    host, action = line.split(/s+/, 2)
    if action == "OK"
      whitelist[host] = true
    end
  end

  # Check host name with whitelist.
  define_applicable_condition("Whitelist") do |condition|
    condition.description = "Whitelist"

    condition.define_connect_stopper do |context, host, address|
      whitelist[host]
    end
  end

  # Apply whitelist to all defined milters.
  defined_milters.each do |name|
    define_milter(name) do |milter|
      milter.add_applicable_condition("Whitelist")
    end
  end


It seems that access(5) format support is useful.
I'll add access(5) support to milter manager in the next
stable release.

Thanks for your comment.
I got a good idea to improve milter manager. :-)


Thanks,
--
kou
Reply | Threaded
Open this post in threaded view
|

Re: how to bypass milters, whitelist hosts

martin f krafft-2
also sprach Kouhei Sutou <[hidden email]> [2009.05.25.1254 +0200]:
> What format are you using for whitelist?
[...]
> It seems that access(5) format support is useful.
> I'll add access(5) support to milter manager in the next
> stable release.

cidr_table(5) would make more sense.

--
martin | http://madduck.net/ | http://two.sentenc.es/
 
"der beruf ist eine schutzwehr, hinter welche man sich erlaubterweise
 zurückziehen kann, wenn bedenken und sorgen allgemeiner art einen
 anfallen."
                                                 - friedrich nietzsche
 
spamtraps: [hidden email]

digital_signature_gpg.asc (204 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: header/body_checks as end-of-data checks (was: how to bypass milters, whitelist hosts)

Wietse Venema
In reply to this post by martin f krafft-2
martin f krafft:

> also sprach Wietse Venema <[hidden email]> [2009.05.23.1442 +0200]:
> > Before making architectural recommendations, it would help to step
> > back into the reality of how policy servers and milters work.  For
> > one thing, policy servers don't handle message content, and for
> > another, Milters must be able to see every SMTP protocol step.
>
> What about header/body_checks? Those would make sense in
> smtpd_end_of_data_restrictions, wouldn't they? What am I missing to
> understand why they aren't? Is it the fact that those checks should
> apply to all mail processed by postfix, whether received by smtpd or
> not?

Presently, header and body processing are not implemented in the
SMTP server.  This is the consequence of an early implementation
decision, based on a) the observation that not all mail is SMTP
mail and b) the requirement that Postfix had to be as performant
as the competition.  http://www.postfix.org/OVERVIEW.html has an
overview of what Postfix functions are performed where.

If you have suggestions for Postfix's user interface or internals,
then it is up to you to do a more significant portion of the analysis
than simply tossing wild ideas into the mailing list. I don't like
discussions where I have to put in 90% of the work.

        Wietse