more actions for *header_checks/body_checks

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

more actions for *header_checks/body_checks

Solar Designer
Hi,

I'd like to have an ACCEPT action for *header_checks and body_checks.
This was requested before:

http://archives.neohapsis.com/archives/postfix/2005-02/1116.html

I've read the replies in that thread, and I continue to think that
adding an ACCEPT action for *header_checks and body_checks is a good
idea - for whitelisting mail based on individual headers or message
body lines.  Specifically, I'd like to whitelist PGP-encrypted mail on a
certain Postfix server, and going for an SMTP proxy or Milter based
filter sounds like overkill for that should-be trivial task.  I'd use
ACCEPT in body_checks if it were available.  I almost feel like adding
ACCEPT support to Postfix would be quicker for me to do than going for
those external filters.

Am I missing some other easy way to do it?

Also nice to have:

- Ability to specify a default action (e.g., be able to use
default-REJECT for header_checks, then ACCEPT only messages with lines
matching one of the patterns).

- A fix for this documented limitation (in BUGS): "A decision made for
one line is not carried over to the next line."  It could be fixed e.g.
by adding a MARK action (similar to its meaning with iptables) and
ability to match based on line regexp in conjunction with such marks.
Then ability to negate a match ("no mark" and/or "regexp not matched")
would also be needed.

Any comments?

Thanks,

Alexander
Reply | Threaded
Open this post in threaded view
|

Re: more actions for *header_checks/body_checks

Noel Jones-2
On 11/16/2011 8:32 AM, Solar Designer wrote:
> Hi,
>
> I'd like to have an ACCEPT action for *header_checks and body_checks.
> This was requested before:
>
> http://archives.neohapsis.com/archives/postfix/2005-02/1116.html
>
> I've read the replies in that thread, and I continue to think that
> adding an ACCEPT action for *header_checks and body_checks is a good

This is unlikely to happen.  header_checks are intended for
site-safe filtering.  If you need more fine-grained control, use eg.
SpamAssassin.

State, including what message the line belongs to, is not saved
between lines.

Adding any kind of whole-message action would require major changes
to the way cleanup works, and is unlikely to happen anytime soon.


  -- Noel Jones
Reply | Threaded
Open this post in threaded view
|

Re: more actions for *header_checks/body_checks

Solar Designer
On Wed, Nov 16, 2011 at 09:36:21AM -0600, Noel Jones wrote:

> On 11/16/2011 8:32 AM, Solar Designer wrote:
> > I'd like to have an ACCEPT action for *header_checks and body_checks.
> > This was requested before:
> >
> > http://archives.neohapsis.com/archives/postfix/2005-02/1116.html
> >
> > I've read the replies in that thread, and I continue to think that
> > adding an ACCEPT action for *header_checks and body_checks is a good
>
> This is unlikely to happen.  header_checks are intended for
> site-safe filtering.

I think that whitelisting of certain messages is site-safe.  It does not
imply that all other messages are rejected (although that is a possible
configuration too and is a reasonable one for some sites/machines/VMs).

> If you need more fine-grained control, use eg. SpamAssassin.

I don't feel that whitelisting of PGP-encrypted messages is more
fine-grained than the kind of blacklisting currently supported in
header_checks and body_checks.

SpamAssassin in particular is too heavy for this trivial task and
machine (256 MB RAM, which is enough for this machine's purpose).
I simply want to define a fairly trivial policy on what's accepted and
what's rejected.  The enhancements that I proposed would be sufficient.

> State, including what message the line belongs to, is not saved
> between lines.
>
> Adding any kind of whole-message action would require major changes
> to the way cleanup works, and is unlikely to happen anytime soon.

I admit I'm not familiar with the code and I haven't tried to implement
ACCEPT yet, but aren't DISCARD and REJECT also whole-message actions?
Is ACCEPT somehow very different?

Thank you for your comments!

Alexander
Reply | Threaded
Open this post in threaded view
|

Re: more actions for *header_checks/body_checks

/dev/rob0
On Wednesday 16 November 2011 10:06:36 Solar Designer wrote:

> On Wed, Nov 16, 2011 at 09:36:21AM -0600, Noel Jones wrote:
> > If you need more fine-grained control, use eg. SpamAssassin.
>
> I don't feel that whitelisting of PGP-encrypted messages is more
> fine-grained than the kind of blacklisting currently supported in
> header_checks and body_checks.
>
> SpamAssassin in particular is too heavy for this trivial task and
> machine (256 MB RAM, which is enough for this machine's purpose).
> I simply want to define a fairly trivial policy on what's accepted
> and what's rejected.  The enhancements that I proposed would be
> sufficient.
>
> > State, including what message the line belongs to, is not saved
> > between lines.
> >
> > Adding any kind of whole-message action would require major
> > changes to the way cleanup works, and is unlikely to happen
> > anytime soon.
>
> I admit I'm not familiar with the code and I haven't tried to
> implement ACCEPT yet, but aren't DISCARD and REJECT also
> whole-message actions? Is ACCEPT somehow very different?

Yes. What do you think this ACCEPT action would do, unconditionally
permit the mail regardless of other reject actions elsewhere? This
isn't possible given Postfix design.

A permit action only applies in the context where called. A permit
result in smtpd_client_restrictions has no influence over what might
happen in smtpd_recipient_restrictions, for example.

A single reject action anywhere before acceptance causes the mail to
be rejected. Numerous permit (or dunno) actions are required for
acceptance; one per restriction stage, one per each header evaluated
in header_checks(5), one for each line in body_checks(5).

If you really think this is worth pursuing, you should consider either
content filtering (inefficient, not safe) or a different MTA.
--
    Offlist mail to this address is discarded unless
    "/dev/rob0" or "not-spam" is in Subject: header
Reply | Threaded
Open this post in threaded view
|

Re: more actions for *header_checks/body_checks

Wietse Venema
In reply to this post by Solar Designer
Solar Designer:
[on whitelisting]
> > State, including what message the line belongs to, is not saved
> > between lines.
> >
> > Adding any kind of whole-message action would require major changes
> > to the way cleanup works, and is unlikely to happen anytime soon.
>
> I admit I'm not familiar with the code and I haven't tried to implement
> ACCEPT yet, but aren't DISCARD and REJECT also whole-message actions?
> Is ACCEPT somehow very different?

Hey, Alexander. Right now, DISCARD / REJECT are easy to explain
(Postfix does not receive the message) and easy to implement (the
cleanup server simply ignores all input until end-of-message and
returns an appropriate status).

ACCEPT is simply to explain only if it disables all further checks.
Things become messy otherwise.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: more actions for *header_checks/body_checks

/dev/rob0
In reply to this post by /dev/rob0
On Wednesday 16 November 2011 10:39:00 I wrote:
> A single reject action anywhere before acceptance causes the mail
> to be rejected. Numerous permit (or dunno) actions are required
> for acceptance; one per restriction stage, one per each header
> evaluated in header_checks(5), one for each line in
> body_checks(5).

Strictly speaking, there must be a dunno/permit for each individual
restriction in each stage. Order matters, of course; a permit action
terminates the processing of further restrictions in that stage. But
in this regard a header_checks rule is like a restriction stage unto
itself.
--
    Offlist mail to this address is discarded unless
    "/dev/rob0" or "not-spam" is in Subject: header
Reply | Threaded
Open this post in threaded view
|

Re: more actions for *header_checks/body_checks

Solar Designer
In reply to this post by /dev/rob0
On Wed, Nov 16, 2011 at 10:39:00AM -0600, /dev/rob0 wrote:
> On Wednesday 16 November 2011 10:06:36 Solar Designer wrote:
> > I admit I'm not familiar with the code and I haven't tried to
> > implement ACCEPT yet, but aren't DISCARD and REJECT also
> > whole-message actions? Is ACCEPT somehow very different?
>
> Yes. What do you think this ACCEPT action would do, unconditionally
> permit the mail regardless of other reject actions elsewhere?

No, it would merely bypass checks of further input lines against the
same table.

In other words, it would be similar to DUNNO, with the difference being
that further input lines are not inspected.

Alexander
Reply | Threaded
Open this post in threaded view
|

Re: more actions for *header_checks/body_checks

Solar Designer
In reply to this post by Wietse Venema
On Wed, Nov 16, 2011 at 11:48:34AM -0500, Wietse Venema wrote:
> ACCEPT is simply to explain only if it disables all further checks.
> Things become messy otherwise.

Maybe we should call it other than ACCEPT, then - to make it clear that
other restrictions elsewhere may still reject the message?  Would that
be simple enough to explain if we pick a proper name and description in
the man page?  I think so.

Some names to consider: PERMIT, BYPASS, DONE, STOP, EXIT.

I think that PERMIT more clearly indicates that we're talking about
permission from this specific check, not acceptance of the message.

In a sense, I need a DUNNO2, but that's probably not a good name.

Alexander
Reply | Threaded
Open this post in threaded view
|

Re: more actions for *header_checks/body_checks

Wietse Venema
In reply to this post by Solar Designer
Solar Designer:

> On Wed, Nov 16, 2011 at 10:39:00AM -0600, /dev/rob0 wrote:
> > On Wednesday 16 November 2011 10:06:36 Solar Designer wrote:
> > > I admit I'm not familiar with the code and I haven't tried to
> > > implement ACCEPT yet, but aren't DISCARD and REJECT also
> > > whole-message actions? Is ACCEPT somehow very different?
> >
> > Yes. What do you think this ACCEPT action would do, unconditionally
> > permit the mail regardless of other reject actions elsewhere?
>
> No, it would merely bypass checks of further input lines against the
> same table.

You mean:

    header_checks = xx:xx, yy:yy, zz:zz

There are several problems with this idea.

Some background first. Many Postfix table lookups are handled by
three layers of code:

1) Table-driven mechanisms (header_checks, virtual_alias_maps,
transport_maps). This layer knows what ACCEPT means in header_checks,
but it has no information about the table that provided the result.

2) Drivers for multi-table searches. This driver has no clue about
header_checks or transport_maps, it has no clue what the lookup
result means, and it does not record any information about the table
that provided a match, because it has never needed to worry about
such things.

3) Table lookup mechanisms (hash, regexp, pcre, ldap, tcp). These
have no clue about header_checks or transport_maps, and they have
no clue what the lookup result means.

> In other words, it would be similar to DUNNO, with the difference being
> that further input lines are not inspected.

So the second header_checks table can ACCEPT the message, but the
first header_checks table can still REJECT it upon a later input.
I expect usability problems with this approach.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: more actions for *header_checks/body_checks

Wietse Venema
In reply to this post by Solar Designer
Solar Designer:

> On Wed, Nov 16, 2011 at 11:48:34AM -0500, Wietse Venema wrote:
> > ACCEPT is simply to explain only if it disables all further checks.
> > Things become messy otherwise.
>
> Maybe we should call it other than ACCEPT, then - to make it clear that
> other restrictions elsewhere may still reject the message?  Would that
> be simple enough to explain if we pick a proper name and description in
> the man page?  I think so.
>
> Some names to consider: PERMIT, BYPASS, DONE, STOP, EXIT.
>
> I think that PERMIT more clearly indicates that we're talking about
> permission from this specific check, not acceptance of the message.
>
> In a sense, I need a DUNNO2, but that's probably not a good name.

I think that the first hurdle is to come up with a model that people
on the list can understand, including those whose first language
skills are much better than their English skills.

I must confess that I no longer understand what the purpose is of
ACCEPT in header_checks, if the purpose is other than skipping
all further lookups of all header_checks tables.

An ACCEPT in header_checks can't extend beyond header_checks's
jurisdiction, just like a permit in smtpd_client_restrictions has
no jurisdiction over smtpd_end_of_data_restrictions.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: more actions for *header_checks/body_checks

Solar Designer
Wietse,

Thank you for your comments and explanation!

On Wed, Nov 16, 2011 at 01:04:10PM -0500, Wietse Venema wrote:
> I must confess that I no longer understand what the purpose is of
> ACCEPT in header_checks, if the purpose is other than skipping
> all further lookups of all header_checks tables.

Actually, I think both kinds of ACCEPTs would make sense in different
circumstances: skip further lookups against the current table and skip
further lookups for all header_checks (and body_checks) tables.  So that
would be two action names.  Hmm, yes, this does become confusing, unless
we use lengthy names like SKIP_THIS_TABLE or SKIP_ALL_TABLES.

More importantly, I realized that the two things that I mentioned as
nice to have - configurable default action per-table and MARK action
(with marks persistent across table boundaries) - are actually almost
required to make the new ACCEPT action (or whatever we call it) usable
in more than just some special cases.  Having MARK may also eliminate
the need for the SKIP_ALL_TABLES variety.

Specifically, a persistent mark is needed to be able to encode logical
expressions involving matches against both headers and body lines -
e.g., permit all mail to these addresses (header check), but only
PGP-encrypted to other addresses (body check to be applied only when the
header check did not match).  I think this could also be achieved with
SKIP_ALL_TABLES (if its uses in a headers_check table would also skip
body_checks), but MARK is more generic.

This does become complicated, yet maybe it could be implemented with
little code while making Postfix's built-in filtering a lot more capable.

Thanks again,

Alexander
Reply | Threaded
Open this post in threaded view
|

Re: more actions for *header_checks/body_checks

Wietse Venema
Solar Designer:

> Wietse,
>
> Thank you for your comments and explanation!
>
> On Wed, Nov 16, 2011 at 01:04:10PM -0500, Wietse Venema wrote:
> > I must confess that I no longer understand what the purpose is of
> > ACCEPT in header_checks, if the purpose is other than skipping
> > all further lookups of all header_checks tables.
>
> Actually, I think both kinds of ACCEPTs would make sense in different
> circumstances: skip further lookups against the current table and skip
> further lookups for all header_checks (and body_checks) tables.  So that
> would be two action names.  Hmm, yes, this does become confusing, unless
> we use lengthy names like SKIP_THIS_TABLE or SKIP_ALL_TABLES.

As I explained in a different response, there is no "this table"
concept outside the low-level (pcre, hash, etc.) table itself.  At
higher levels, there is a lookup result without source attribution.

Considering Postfix's drive to economy of mechanisms, a "this table"
concept at higher levels (e.g. multi-table driver, or even higher)
is unlikely.

So all we can do at the moment is an ACCEPT operation that skips
all further lookups in all header_checks tables.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: more actions for *header_checks/body_checks

Solar Designer
On Wed, Nov 16, 2011 at 04:11:17PM -0500, Wietse Venema wrote:

> As I explained in a different response, there is no "this table"
> concept outside the low-level (pcre, hash, etc.) table itself.  At
> higher levels, there is a lookup result without source attribution.
>
> Considering Postfix's drive to economy of mechanisms, a "this table"
> concept at higher levels (e.g. multi-table driver, or even higher)
> is unlikely.
>
> So all we can do at the moment is an ACCEPT operation that skips
> all further lookups in all header_checks tables.
OK.  I took a look at the code and I see those difficulties now.  How
about something like the attached patch?  It's totally untested other
than that it compiles, and it's probably wrong (especially considering
that it's the first time I am dealing with this code) - but I think it
illustrates what I am speaking about.

The patch tries to implement what you wrote above plus a default action.
I think it (once tested, debugged, and fixed) should let me set the
default action to REJECT, then change it to ACCEPT in a header_checks
table for certain target addresses and also to ACCEPT in body_checks for
PGP-encrypted messages regardless of target address.  If so, this will
suit my needs right now.

Alexander

postfix-2.8.7-hbc-default-and-accept.diff (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: more actions for *header_checks/body_checks

Wietse Venema
Solar Designer:

> On Wed, Nov 16, 2011 at 04:11:17PM -0500, Wietse Venema wrote:
> > As I explained in a different response, there is no "this table"
> > concept outside the low-level (pcre, hash, etc.) table itself.  At
> > higher levels, there is a lookup result without source attribution.
> >
> > Considering Postfix's drive to economy of mechanisms, a "this table"
> > concept at higher levels (e.g. multi-table driver, or even higher)
> > is unlikely.
> >
> > So all we can do at the moment is an ACCEPT operation that skips
> > all further lookups in all header_checks tables.
>
> OK.  I took a look at the code and I see those difficulties now.  How
> about something like the attached patch?  It's totally untested other
> than that it compiles, and it's probably wrong (especially considering
> that it's the first time I am dealing with this code) - but I think it
> illustrates what I am speaking about.

It appears to disable body_checks and perhaps Milters too. This is
not hard to fix.  The idea of redirecting header callback to a NOOP
is interesting (but this needs to be part of the per-message state,
as it must not affect the next message).

I think it is not a problem to add an ACCEPT this message action
now. You don't have to provide the whole solution.

        Wietse

> The patch tries to implement what you wrote above plus a default action.
> I think it (once tested, debugged, and fixed) should let me set the
> default action to REJECT, then change it to ACCEPT in a header_checks
> table for certain target addresses and also to ACCEPT in body_checks for
> PGP-encrypted messages regardless of target address.  If so, this will
> suit my needs right now.
>
> Alexander
Reply | Threaded
Open this post in threaded view
|

Re: more actions for *header_checks/body_checks

Solar Designer
On Wed, Nov 16, 2011 at 08:02:03PM -0500, Wietse Venema wrote:
> Solar Designer:
> > OK.  I took a look at the code and I see those difficulties now.  How
> > about something like the attached patch?  It's totally untested other
> > than that it compiles, and it's probably wrong (especially considering
> > that it's the first time I am dealing with this code) - but I think it
> > illustrates what I am speaking about.
>
> It appears to disable body_checks

On purpose - after a match resulting in ACCEPT, that is.  I admit that I
had my specific use case in mind, though.

> and perhaps Milters too.

Yes, it looks so.  Instead of CLEANUP_FLAG_FILTER_ALL I think it should
use CLEANUP_FLAG_FILTER.

> This is
> not hard to fix.  The idea of redirecting header callback to a NOOP
> is interesting (but this needs to be part of the per-message state,
> as it must not affect the next message).

DISCARD does the same thing with flags, so I assumed it was per-message.
No?

> I think it is not a problem to add an ACCEPT this message action
> now. You don't have to provide the whole solution.

Does this mean you're going to implement it?  Sounds great if so.  And
the default action feature, please - I'd use them together.

In my demo patch, I apply the default action in all not-ACCEPT cases,
but this will need to be restricted to have the default action applied
when no other action was found.  Also, the default action name could be
checked against a list of reasonable ones (report an error to the
Postfix install admin if not).

Thanks,

Alexander
Reply | Threaded
Open this post in threaded view
|

Re: more actions for *header_checks/body_checks

Wietse Venema
Solar Designer:

> On Wed, Nov 16, 2011 at 08:02:03PM -0500, Wietse Venema wrote:
> > Solar Designer:
> > > OK.  I took a look at the code and I see those difficulties now.  How
> > > about something like the attached patch?  It's totally untested other
> > > than that it compiles, and it's probably wrong (especially considering
> > > that it's the first time I am dealing with this code) - but I think it
> > > illustrates what I am speaking about.
> >
> > It appears to disable body_checks
>
> On purpose - after a match resulting in ACCEPT, that is.  I admit that I
> had my specific use case in mind, though.
>
> > and perhaps Milters too.
>
> Yes, it looks so.  Instead of CLEANUP_FLAG_FILTER_ALL I think it should
> use CLEANUP_FLAG_FILTER.
>
> > This is
> > not hard to fix.  The idea of redirecting header callback to a NOOP
> > is interesting (but this needs to be part of the per-message state,
> > as it must not affect the next message).
>
> DISCARD does the same thing with flags, so I assumed it was per-message.
> No?
>
> > I think it is not a problem to add an ACCEPT this message action
> > now. You don't have to provide the whole solution.
>
> Does this mean you're going to implement it?  Sounds great if so.  And
> the default action feature, please - I'd use them together.

ACCEPT in header_checks == turn off header checks for this message.

No Postfix table-driven feature has support for unmatched patterns;
No Postfix table-driven "yes/permit/accept" feature overrides other
table-driven features.

If you really want such things then I suggest using a Perl script
with Net::SMTP as a tiny content filter.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: more actions for *header_checks/body_checks

Solar Designer
On Thu, Nov 17, 2011 at 06:39:29AM -0500, Wietse Venema wrote:
> Solar Designer:
> > Does this mean you're going to implement it?  Sounds great if so.  And
> > the default action feature, please - I'd use them together.
>
> ACCEPT in header_checks == turn off header checks for this message.

Right.  (And maybe body checks as well, or maybe this contradicts the
policy stated below...)

> No Postfix table-driven feature has support for unmatched patterns;
> No Postfix table-driven "yes/permit/accept" feature overrides other
> table-driven features.

I suppose adhering to this policy has both pros and cons.

What about something like this? -

/malware sig with occasional false positives/ DEFERRED_REJECT malware detected
/whitelisted sender address/ ACCEPT

where DEFERRED_REJECT would alter the default action for the current
message and ACCEPT would override that?  It'd work as desired (the
whitelisting would take priority) regardless of the order in which the
two patterns are seen in the headers or body.  After the headers and
body are fully processed, we'd have the per-message default action set
to DEFERRED_REJECT, but it would only be applied if the ACCEPT flag is
not set.  We would also have DEFERRED_DISCARD, indeed.

> If you really want such things then I suggest using a Perl script
> with Net::SMTP as a tiny content filter.

Yes, this (or something very much like it) is my primary alternative.
The reason why I brought this to postfix-users instead of just doing
things on the specific system in one way or another was that I felt we
could improve Postfix for others as well.

Alexander
Reply | Threaded
Open this post in threaded view
|

Re: more actions for *header_checks/body_checks

Wietse Venema
Solar Designer:

> On Thu, Nov 17, 2011 at 06:39:29AM -0500, Wietse Venema wrote:
> > Solar Designer:
> > > Does this mean you're going to implement it?  Sounds great if so.  And
> > > the default action feature, please - I'd use them together.
> >
> > ACCEPT in header_checks == turn off header checks for this message.
>
> Right.  (And maybe body checks as well, or maybe this contradicts the
> policy stated below...)
>
> > No Postfix table-driven feature has support for unmatched patterns;
> > No Postfix table-driven "yes/permit/accept" feature overrides other
> > table-driven features.
>
> I suppose adhering to this policy has both pros and cons.

Let's for a monent look at what the system does well and why. The
system is designed for blocking stuff. There are N rule sets that
can block/discard independently (smtpd_client_restrictions etc.,
header_checks etc.,). If any of them blocks, mail is not queued.
It's a vetoing system. All the other actions such as filter or
redirect are just riding along on the train.

This model is a good fit for vetoing. If is a poor fit for lots of
other things including 1) conditional decisions and 2) "don't reject"
decisions that override other rulesets. So let's not do that.

You have a need for conditional decisions, which could be modeled
as a "maybe reject" action that only takes effect if some other
action (accept) in the same rule set does not fire.

    But why stop here? Why not have "maybe filter", "maybe redirect",
    and so on that are also dependent on the absence of some other
    action (what other action?  OK, so we do "ifnot accept reject",
    "ifnot filter redirect", etc. to make the condition explicit.
    Yes, it is possible. No, it is not a good idea.  The resulting
    system would be an incomprehensible mess.

In addition you have a need for a positive decision (already a poor
fit for a vetoing system!) in one rule set to override decisions of
other rulesets (here, an accept in header_checks that overrides
rule sets that happen to be co-located in the cleanup daemon).

I'm not saying that your needs are wrong. I am saying that you
need to use a model that is a good fit. It would be a mistake to
change a model that is a poor fit, such as a veto-based system.

It is very well possible that your needs require a model with rules
that have precedences. For example, one receives the entire message,
and applies all the rules in order of precedence and stops at the
first final decision (ACCEPT or REJECT).

Postfix does not work that way. It does not have to solve all
problems. That is what the extension interfaces (including SMTP)
are for.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: more actions for *header_checks/body_checks

Solar Designer
Wietse -

Thank you for explaining your position on this in so much detail!  You
definitely have a pretty strong and reasonable opinion here.  I was
essentially after mail filtering capabilities analogous to those of a
network packet filter and I felt that those would reasonably fit into
Postfix proper, but indeed I accept your opinion that they don't.

I've included some comments below.  These are not meant to argue with
you, but merely to communicate my thoughts on the matter, even though
these probably do not matter as you've already made the determination.

On Thu, Nov 17, 2011 at 11:38:02AM -0500, Wietse Venema wrote:
> Let's for a monent look at what the system does well and why. The
> system is designed for blocking stuff. There are N rule sets that
> can block/discard independently (smtpd_client_restrictions etc.,
> header_checks etc.,). If any of them blocks, mail is not queued.
> It's a vetoing system. All the other actions such as filter or
> redirect are just riding along on the train.

Agreed.

> This model is a good fit for vetoing. If is a poor fit for lots of
> other things including 1) conditional decisions and 2) "don't reject"
> decisions that override other rulesets.

Yes.

> So let's not do that.

OK, yet I felt that we could do a little bit of that at one specific
layer only.

> You have a need for conditional decisions, which could be modeled
> as a "maybe reject" action that only takes effect if some other
> action (accept) in the same rule set does not fire.

Right.

>     But why stop here? Why not have "maybe filter", "maybe redirect",
>     and so on that are also dependent on the absence of some other
>     action (what other action?  OK, so we do "ifnot accept reject",
>     "ifnot filter redirect", etc. to make the condition explicit.
>     Yes, it is possible. No, it is not a good idea.  The resulting
>     system would be an incomprehensible mess.

Yes, I had these thoughts/concerns, too.  I think I'd stop at supporting
those other default actions, but not adding the if/ifnot logic at that
level.  For such logic, if desired, I had proposed MARK actions and
matching on marks and not-marks.  This would be similar to the way in
which and to extent to which network packet filters may be configured.

Individual header/body lines may be analogous to individual network
packets, and messages to network connections.  State from line (packet)
matches may be passed onto decisions for messages (connections).

I felt that this would not be too hard and too messy to implement on top
of what we readily have in Postfix.

> In addition you have a need for a positive decision (already a poor
> fit for a vetoing system!) in one rule set to override decisions of
> other rulesets (here, an accept in header_checks that overrides
> rule sets that happen to be co-located in the cleanup daemon).

I feel that header_checks and body_checks are very closely related to
each other (and share code).  I think it could be reasonable to consider
them one ruleset, within which this packet filter like logic would be
present.  In fact, they're even documented in just one man page.

> I'm not saying that your needs are wrong. I am saying that you
> need to use a model that is a good fit. It would be a mistake to
> change a model that is a poor fit, such as a veto-based system.

Maybe, but it's tempting to enhance the existing model if it can be done
with little complexity to the code.

> It is very well possible that your needs require a model with rules
> that have precedences. For example, one receives the entire message,
> and applies all the rules in order of precedence and stops at the
> first final decision (ACCEPT or REJECT).

I don't have an immediate need for that, but, yes, it would arguably
resemble a network packet filter more closely.  On the other hand, with
the messages-connections analogy, I'd have to use MARKs either way and
not care about the order of lines-packets.

> Postfix does not work that way. It does not have to solve all
> problems. That is what the extension interfaces (including SMTP)
> are for.

I definitely agree that Postfix should not (and can not) solve all
problems.  The question is where to stop.  I felt that minor additions
to the existing feature set and code, yet that would make the built-in
filtering sufficient in a lot more cases, would be appropriate.  You
have pointed out that this would cross some boundaries unrelated to
Postfix code complexity.  I accept this.

Thanks again,

Alexander