Small Enhancement for the Policy Server Protocol

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Small Enhancement for the Policy Server Protocol

Ronald F. Guilmette-2

I'd like to propose a small enhancement for the Policy Server protocol.
I'll code up a first cut of it, if nobody else is willing.

Basically, I think it would be very useful if the protcol included a
line like:

        trusted_client=[yes/no]

where the value would be set to "yes" if and only if the client had
either authenticated (SASL) _or_ the client connected from some IP
specified in $my_networks.

Rationale:

Handling of the e-mails processed by Postfix could be broadly grouped into
"incoming" and "outgoing" categories.  The handling of each category might
reasonably be very different.  For example, while it may be desirable to
apply all sorts of filtering tests to incoming e-mail, one probably does
not want to apply such tests to outgoing e-mails.  Separately, if one
wishes to automatically construct and/or maintain an whitelist of every
e-mail address that "internal" (trusted) users have sent mail to in the
past, then one could do that by simply adding all recipient e-mail
addresses associated with _just_ "outgoing" e-mails to the automatically
maintained whitelist.  (Such a whitelist could be easily maintained in an
entirely automated fasion by a Postfix policy server, but the policy server
has to differentiate between incoming and outgoing e-mails.)

I've already written something (a policy server) that does in fact try to
maintain a system-wide whitelist in just this way, but it is less than
optimal because it has to check itself to see if either (a) there is one
or more sasl_* lines in the current policy request or else (b) the client
IP is in $my_networks.  For the latter test, the tool in question has to
be provided with its own copy of the value of $my_networks (because it is
not smart enough to know how to parse this data out of Postfix's *.cf files)
and I'm worried that this will eventually lead to a "version skew" type
problem, i.e. where Postfix has one version of the $my_networks list, but
the external (whitelist maintainer) policy server has a somewhat different
version of $my_networks, in which case proper maintenance of the whitelist
will break down.

Given that Postfix is already checking to see if the client is in $my_networks
anyway, _and_ that it already knows whether or not the current SMTP session
has authenticated (via SASL), it seems that it would be quite simple for
Postfix to simply share this bit of useful information (in a convenient form)
with external policy servers.

Reply | Threaded
Open this post in threaded view
|

Re: Small Enhancement for the Policy Server Protocol

Wietse Venema
Ronald F. Guilmette:

>
> I'd like to propose a small enhancement for the Policy Server protocol.
> I'll code up a first cut of it, if nobody else is willing.
>
> Basically, I think it would be very useful if the protcol included a
> line like:
>
> trusted_client=[yes/no]
>
> where the value would be set to "yes" if and only if the client had
> either authenticated (SASL) _or_ the client connected from some IP
> specified in $my_networks.

You are assuming that mynetworks, SASL and TLS will grant identical
privileges to the client, but there is nothing that requires this
to be so.

For this reason, combining multiple different things in one name
is not a good idea. Perhaps it is better to extend the existing
sasl_ and cert_ attributes where needed (for example, sasl_method
and sasl_username already tell you if someone used SASL successfully).

        Wietse

> Rationale:
>
> Handling of the e-mails processed by Postfix could be broadly grouped into
> "incoming" and "outgoing" categories.  The handling of each category might
> reasonably be very different.  For example, while it may be desirable to
> apply all sorts of filtering tests to incoming e-mail, one probably does
> not want to apply such tests to outgoing e-mails.  Separately, if one
> wishes to automatically construct and/or maintain an whitelist of every
> e-mail address that "internal" (trusted) users have sent mail to in the
> past, then one could do that by simply adding all recipient e-mail
> addresses associated with _just_ "outgoing" e-mails to the automatically
> maintained whitelist.  (Such a whitelist could be easily maintained in an
> entirely automated fasion by a Postfix policy server, but the policy server
> has to differentiate between incoming and outgoing e-mails.)
>
> I've already written something (a policy server) that does in fact try to
> maintain a system-wide whitelist in just this way, but it is less than
> optimal because it has to check itself to see if either (a) there is one
> or more sasl_* lines in the current policy request or else (b) the client
> IP is in $my_networks.  For the latter test, the tool in question has to
> be provided with its own copy of the value of $my_networks (because it is
> not smart enough to know how to parse this data out of Postfix's *.cf files)
> and I'm worried that this will eventually lead to a "version skew" type
> problem, i.e. where Postfix has one version of the $my_networks list, but
> the external (whitelist maintainer) policy server has a somewhat different
> version of $my_networks, in which case proper maintenance of the whitelist
> will break down.
>
> Given that Postfix is already checking to see if the client is in $my_networks
> anyway, _and_ that it already knows whether or not the current SMTP session
> has authenticated (via SASL), it seems that it would be quite simple for
> Postfix to simply share this bit of useful information (in a convenient form)
> with external policy servers.
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Small Enhancement for the Policy Server Protocol

Ronald F. Guilmette-2

In message <[hidden email]>,
[hidden email] (Wietse Venema) wrote:

>Ronald F. Guilmette:
>>
>> I'd like to propose a small enhancement for the Policy Server protocol.
>> I'll code up a first cut of it, if nobody else is willing.
>>
>> Basically, I think it would be very useful if the protcol included a
>> line like:
>>
>> trusted_client=[yes/no]
>>
>> where the value would be set to "yes" if and only if the client had
>> either authenticated (SASL) _or_ the client connected from some IP
>> specified in $my_networks.
>
>You are assuming that mynetworks, SASL and TLS will grant identical
>privileges to the client

I don't believe that I was making such an assumption, but in any case,
I think the rest of what you said may make that debate moot anyay.

>For this reason, combining multiple different things in one name
>is not a good idea.

As a general design principal, I would have to agree with you 100% on that.
But of course, there are always times when one has to be a bit flexible about
one's design principals, in order to achieve the objective most succinctly
(e.g. reject_unauth_destination, which combines multiple kinds of checks
into a single name).  But here again, we could save this philosophoical
debate for another day, because I think what you proposed (below) will
solve my original problem neatly, and it wouldn't violate your design
principal.

>Perhaps it is better to extend the existing
>sasl_ and cert_ attributes where needed

OK.  That would work for me.  Could I just have something like this?

        client_in_my_networks=[yes/no]

>(for example, sasl_method
>and sasl_username already tell you if someone used SASL successfully).

Yes.  Quite so.  So that takes care of the SASL part of the problem.  Now
I'd just like my policy server to have enough information so that it can
also properly deal with the case where the client is trused on account of
where he is, not on account of his having been formally authenticated.

Reply | Threaded
Open this post in threaded view
|

Re: Small Enhancement for the Policy Server Protocol

Wietse Venema
Ronald F. Guilmette:

>
> In message <[hidden email]>,
> [hidden email] (Wietse Venema) wrote:
>
> >Ronald F. Guilmette:
> >>
> >> I'd like to propose a small enhancement for the Policy Server protocol.
> >> I'll code up a first cut of it, if nobody else is willing.
> >>
> >> Basically, I think it would be very useful if the protcol included a
> >> line like:
> >>
> >> trusted_client=[yes/no]
> >>
> >> where the value would be set to "yes" if and only if the client had
> >> either authenticated (SASL) _or_ the client connected from some IP
> >> specified in $my_networks.
> >
> >You are assuming that mynetworks, SASL and TLS will grant identical
> >privileges to the client
>
> I don't believe that I was making such an assumption, but in any case,
> I think the rest of what you said may make that debate moot anyay.
>
> >For this reason, combining multiple different things in one name
> >is not a good idea.
>
> As a general design principal, I would have to agree with you 100% on that.
> But of course, there are always times when one has to be a bit flexible about
> one's design principals, in order to achieve the objective most succinctly
> (e.g. reject_unauth_destination, which combines multiple kinds of checks
> into a single name).  But here again, we could save this philosophoical
> debate for another day, because I think what you proposed (below) will
> solve my original problem neatly, and it wouldn't violate your design
> principal.
>
> >Perhaps it is better to extend the existing
> >sasl_ and cert_ attributes where needed
>
> OK.  That would work for me.  Could I just have something like this?
>
> client_in_my_networks=[yes/no]

That might work (under a better name) but it should not encourage
requests to simply dump all the low-level Postfix predicates in
the policy protocol:

    permit_mynetworks=yes
    reject_unauth_destination=no
    reject_rhsbl_sender=yes
    reject_non_fdqn_helo=no

> >(for example, sasl_method
> >and sasl_username already tell you if someone used SASL successfully).
>
> Yes.  Quite so.  So that takes care of the SASL part of the problem.  Now
> I'd just like my policy server to have enough information so that it can
> also properly deal with the case where the client is trused on account of
> where he is, not on account of his having been formally authenticated.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Small Enhancement for the Policy Server Protocol

Ronald F. Guilmette-2

In message <[hidden email]>, Wietse wrote:

>Ronald F. Guilmette:
>> client_in_my_networks=[yes/no]
>
>That might work (under a better name) but it should not encourage
>requests to simply dump all the low-level Postfix predicates in
>the policy protocol:

Well, it's purpose is limited to just what I stated, which is likely to
be a common problem/issue for many folks building policy servers, i.e.
differentiating properly between "incoming" and "outgoing".  So I doubt
that it would be a basis for anybody asking for anything more.  (But
you never know.)

So anyway, should I take a whack at trying to code up an implementation
of this new protocol parameter and then send you some code patches for
review?  Or would you prefer just to take the ball and run with it yourself?
Either way is fine by me.
Reply | Threaded
Open this post in threaded view
|

Re: Small Enhancement for the Policy Server Protocol

Wietse Venema
Ronald F. Guilmette:

>
> In message <[hidden email]>, Wietse wrote:
>
> >Ronald F. Guilmette:
> >> client_in_my_networks=[yes/no]
> >
> >That might work (under a better name) but it should not encourage
> >requests to simply dump all the low-level Postfix predicates in
> >the policy protocol:
>
> Well, it's purpose is limited to just what I stated, which is likely to
> be a common problem/issue for many folks building policy servers, i.e.
> differentiating properly between "incoming" and "outgoing".  So I doubt
> that it would be a basis for anybody asking for anything more.  (But
> you never know.)

I doubt whether one can report in a policy protocol attribute
whether or not a client TLS certificate would grant mail relay or
other privileges; ditto with "pop before SMTP" and other mechanisms.

I also don't think that it makes sense to "bolt on" the concept of
inbound and outbound mail onto Postfix, because it was explicitly
designed not to have such a concept.

When a remote user sends mail to [hidden email], and when a
local porcupine.org user sends mail to that same address, it makes
no sense to call one inbound and the other something different.

Instead of inbound/outbound, Postfix uses the concept of mail relay
authorization in the SMTP server.

If the policy protocol is to provide the information needed to
determine mail relay authorization, then it would have to list all
the configured smtpd_recipient_restrictions as policy protocol
elements.

> So anyway, should I take a whack at trying to code up an implementation
> of this new protocol parameter and then send you some code patches for
> review?  Or would you prefer just to take the ball and run with it yourself?
> Either way is fine by me.

I have no time, but I can proofread and suggest improvements.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Small Enhancement for the Policy Server Protocol

Ronald F. Guilmette-2

In message <[hidden email]>,
[hidden email] (Wietse Venema) wrote:

>Instead of inbound/outbound, Postfix uses the concept of mail relay
>authorization in the SMTP server.

Yes.  Thank you for clarifying.  You're correct that this is really
what I want my policy server to tailor its behavior based upon, i.e.
the answer to the question "Is the current client authorized for
relaying or not?"

>If the policy protocol is to provide the information needed to
>determine mail relay authorization, then it would have to list all
>the configured smtpd_recipient_restrictions as policy protocol
>elements.

Why?

I mean isn't Postfix making this determination (relay authorized / unauthorized)
internally itself already anyway?  Assuming so, then why can't it just simply
pass the results of its own determination out to an external policy server?

I think that you have quite clearly hit the nail on the head when you
say that the information I really want in the policy server is just the
[yes/no] answer to the question "Is the client authorized for relaying?"

It seems to me that Postfix already must compute the answer to that
question, based upon (as you have noted) a whole bunch of different bits
of Postfix-specific configuration information.  Yes?  Assuming so, then
is there anything that would prevent Postfix from simply passing that
one useful bit of information out/down to an external policy server?

Quite obviously, you are correct when you say that Postfix has to take
a whole lot of different variables (actually, configuration parameters)
into account when it is computing the answer to the question:  "Is the
client authorized (for relaying)?"  And the computation itself is somewhat
complex.  This is all the more reason why I really _don't_ want to try to
mirror or duplicate Postfix's own calculations of the "Is the client
authorized?" answer within my own policy server... because I am likely
to screw it up in some cases and not reach the exactly same answer as
Postfix itself reached... at least on some occasions (corner cases?).
Whatever causes that divergence, regadless of whether it is version skew
related to the variables or version skew related to the computation algo-
rithim, if the policy server's computation of the [yes/no] answer reaches
a different result than Postfix itself did, then things are likely to
screw up.  So this is why I think it would be better for Postfix to simply
tell policy servers what it itself already decided, so that ploicy servers
don't have to re-do the analysis themselves, maybe screwing it up in the
process.

>I have no time, but I can proofread and suggest improvements.

OK.  That's fine by me.

Can you please save me a bit of time and give me one hint to get me started
however?  In which data structure (and in which field) does Postfix stash
its own answer to the question "Is the current client relay authorized?"
Would that be struct SMTPD_STATE and field recipient_rcptmap_checked?
Or am I way off base here?  (I just glanced at SMTPD_STATE quickly and
took a wild guess.)
Reply | Threaded
Open this post in threaded view
|

Re: Small Enhancement for the Policy Server Protocol

Wietse Venema
Ronald F. Guilmette:

>
> In message <[hidden email]>,
> [hidden email] (Wietse Venema) wrote:
>
> >Instead of inbound/outbound, Postfix uses the concept of mail relay
> >authorization in the SMTP server.
>
> Yes.  Thank you for clarifying.  You're correct that this is really
> what I want my policy server to tailor its behavior based upon, i.e.
> the answer to the question "Is the current client authorized for
> relaying or not?"
>
> >If the policy protocol is to provide the information needed to
> >determine mail relay authorization, then it would have to list all
> >the configured smtpd_recipient_restrictions as policy protocol
> >elements.
>
> Why?

Because relay authorization is controlled by smtpd_recipient_restrictions.

The decision may involve multiple access controls, including
check_client_access, check_ccert_access, and so on, whose "value"
is currently not available as policy attributes.

Or the decision may involve just one rule:

    smtpd_recipient_restrictions = reject_unauth_destination

which does not even involve mynetworks. This is what I use on
systems that don't relay mail from the network (they can still send
/usr/sbin/sendmail submissions with remote destinations).

> I mean isn't Postfix making this determination (relay authorized
> / unauthorized) internally itself already anyway?

No, the determination is made in smtpd_recipient_restrictions.

> Can you please save me a bit of time and give me one hint to get me started
> however?  In which data structure (and in which field) does Postfix stash
> its own answer to the question "Is the current client relay authorized?"

There is no such field.  smtpd_recipient_restrictions decides if
recipient is accepted or rejected.  Postfix does not know if it was
rejected because of blacklisting or because of lack of permission.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Small Enhancement for the Policy Server Protocol

Ronald F. Guilmette-2

In message <[hidden email]>,
[hidden email] (Wietse Venema) wrote:

>> I mean isn't Postfix making this determination (relay authorized
>> / unauthorized) internally itself already anyway?
>
>No, the determination is made in smtpd_recipient_restrictions.

I won't quibble senamtics.  _Something_ implements/interpolates the
smtpd_recipient_restrictions into a useful result.  That something
is Postfix.

>> Can you please save me a bit of time and give me one hint to get me started
>> however?  In which data structure (and in which field) does Postfix stash
>> its own answer to the question "Is the current client relay authorized?"
>
>There is no such field.  smtpd_recipient_restrictions decides if
>recipient is accepted or rejected.  Postfix does not know if it was
>rejected because of blacklisting or because of lack of permission.

Ah!  OK.  *Now* you have said something that does make me grok the
problem a bit better.  Yes, you're right.  Postfix doesn't actually
have the specific bit of information I want/need in my policy server
(despite the fact that I had really hoped that it did) and thus it
seems I have no choice but implement a separate sort of computation
in the policy server that will give me something which approximates
an incoming/outgoing bit... at least well enough for my specific
practical purposes.

Anyway, thanks for bearing with me long enough for me to actually
understand the dilemma.
Reply | Threaded
Open this post in threaded view
|

Re: Small Enhancement for the Policy Server Protocol

Noel Jones-2
Ronald F. Guilmette wrote:
>  Postfix doesn't actually
> have the specific bit of information I want/need in my policy server
> (despite the fact that I had really hoped that it did) and thus it

When a policy service is called, postfix doesn't know if a
specific client has relay access.  YOU define who has relay
access in smtpd_recipient_restrictions, which may have complex
nested rules, and may not have been evaluated yet.
However, postfix does know if a client is within mynetworks.

It's not unreasonable to add a "mynetworks_client=[yes,no]" to
the policy protocol.

Hack up a patch (with a terse documentation diff too) and
submit it for comment.

--
Noel Jones
Reply | Threaded
Open this post in threaded view
|

Re: Small Enhancement for the Policy Server Protocol

Ronald F. Guilmette-2

In message <[hidden email]>,
Noel Jones <[hidden email]> wrote:

>Ronald F. Guilmette wrote:
>>  Postfix doesn't actually
>> have the specific bit of information I want/need in my policy server
>> (despite the fact that I had really hoped that it did) and thus it
>
>When a policy service is called, postfix doesn't know if a
>specific client has relay access.  YOU define who has relay
>access in smtpd_recipient_restrictions, which may have complex
>nested rules, and may not have been evaluated yet.

Right.  I _did_ grok all that already, based on Wietse's last post.
But thanks for reiterating.

>It's not unreasonable to add a "mynetworks_client=[yes,no]" to
>the policy protocol.

Perhaps, but it does seem rather entirely less useful to me now than it
did earlier.

>Hack up a patch (with a terse documentation diff too) and
>submit it for comment.

I think I'm going to take a pass on that.  But thanks.

It's easy enough to just pass a copy of $mynetworks to an external policy
server, e.g. via the command line argv.
Reply | Threaded
Open this post in threaded view
|

Re: Small Enhancement for the Policy Server Protocol

Noel Jones-2
Ronald F. Guilmette wrote:
> It's easy enough to just pass a copy of $mynetworks to an external policy
> server, e.g. via the command line argv.

#main.cf
mynetworks = /path/to/file
or
mynetworks = cidr:/path/to/file
(the file format is slightly different for each of the above
examples)

use the same file in your policy service.
http://www.postfix.org/postconf.5.html#mynetworks

--
Noel Jones
Reply | Threaded
Open this post in threaded view
|

Re: Small Enhancement for the Policy Server Protocol

Jan P. Kessler-2
Noel Jones schrieb:
> Ronald F. Guilmette wrote:
>> It's easy enough to just pass a copy of $mynetworks to an external
>> policy
>> server, e.g. via the command line argv.


This won't work for policy servers which are not called by postfix
spawn. Maybe it'd be better to exec "postconf mynetworks".

Anyway - all of these approaches like parsing main.cf or calling
"postconf mynetworks" suffer from difficulties in multi instance setups.
And in any case there's a lot of overhead to compute things (fopen, cidr
compare, ...), which have already been determined by postfix. So, I'd
agree to say that sth like 'client_within_mynetworks=(yes/no)' could be
a useful extension to the policy delegation protocol. Other common
smtpd_*_restrictions like tls-* oder sasl_* values are already reflected
by attributes, which allows to use them in a very efficient way - why
not follow the same logic for mynetworks?

Reply | Threaded
Open this post in threaded view
|

Re: Small Enhancement for the Policy Server Protocol

Ronald F. Guilmette-2

In message <[hidden email]>,
"Jan P. Kessler" <[hidden email]> wrote:

>Noel Jones schrieb:
>> Ronald F. Guilmette wrote:
>>> It's easy enough to just pass a copy of $mynetworks to an external
>>> policy
>>> server, e.g. via the command line argv.
>
>
>This won't work for policy servers which are not called by postfix
>spawn. Maybe it'd be better to exec "postconf mynetworks".

Hay!  I like that solution!  Thanks.

>Anyway - all of these approaches like parsing main.cf or calling
>"postconf mynetworks" suffer from difficulties in multi instance setups.
>And in any case there's a lot of overhead to compute things (fopen, cidr
>compare, ...), which have already been determined by postfix. So, I'd
>agree to say that sth like 'client_within_mynetworks=(yes/no)' could be
>a useful extension to the policy delegation protocol.

Well, it did seem to me that it definitely wouldn't hurt.  But then
Wietse pointed out that it might, e.g. because implementing that might
start to drag in lots and lots of other requests for other Postfix
configuration parameters to be passed via the protocol too.  (And he
might be right about that.  I don't really know.)

>Other common
>smtpd_*_restrictions like tls-* oder sasl_* values are already reflected
>by attributes, which allows to use them in a very efficient way - why
>not follow the same logic for mynetworks?

See above.