RFC: Check mail quota at a mail relay (backscatter)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

RFC: Check mail quota at a mail relay (backscatter)

Jozsef Kadlecsik
Hi,

I'd like to prevent to generate bounces at a mail relay due to the
recipient being over the mailbox size limit or quota at the next-hop
destination machine.

Is there a way/would it be feasible to design a Postfix service, which
could be called to query the next-hop destination *before* accepting the
mail? It could be a standard SMTP client issuing always

MAIL FROM:<foo@src> SIZE=xxx
RCPT TO:<bar@dst>

To err on the safe side, let's assume that the restriction corresponding
to the service actually executed if it is called

- from the smtpd_sender|recipient_restrictions and SIZE was announced,
- from the smtpd_data_restriction and SIZE was not announced and
  the message has got a single recipient.

Assuming that the queried next-hop destinations run Postfix as well (;-),
we'd need another service to connect to, which could be a combined simple
smtpd & enhanced local service answering the question whether the message
would fit into the mailbox or disk quota of the recipient.

What do you think? Is it too complicated?

Best regards,
Jozsef
-
E-mail  : [hidden email], [hidden email]
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary
Reply | Threaded
Open this post in threaded view
|

Re: RFC: Check mail quota at a mail relay (backscatter)

mouss-2
Jozsef Kadlecsik wrote:

> Hi,
>
> I'd like to prevent to generate bounces at a mail relay due to the
> recipient being over the mailbox size limit or quota at the next-hop
> destination machine.
>
> Is there a way/would it be feasible to design a Postfix service, which
> could be called to query the next-hop destination *before* accepting the
> mail? It could be a standard SMTP client issuing always
>
> MAIL FROM:<foo@src> SIZE=xxx
> RCPT TO:<bar@dst>
>
> To err on the safe side, let's assume that the restriction corresponding
> to the service actually executed if it is called
>
> - from the smtpd_sender|recipient_restrictions and SIZE was announced,
> - from the smtpd_data_restriction and SIZE was not announced and
>   the message has got a single recipient.
>
> Assuming that the queried next-hop destinations run Postfix as well (;-),
> we'd need another service to connect to, which could be a combined simple
> smtpd & enhanced local service answering the question whether the message
> would fit into the mailbox or disk quota of the recipient.
>  

the remote postfix will not bring much. postfix doesn't know the quota
anyway.
> What do you think?

you can use a policy service called at recipient and data stages. the
service must keep "state" between the calls (so as to "link" the
recipients with the message size). once it knows the size, it can query
a remote daemon (you can use http or whatever protocol you want) to see
whether the user is over quota or not.

note however that if the recipient has mail still queued on the mail
relay, your remote query will miss it. other race conditions may happen
as well. so you should allows for a "little" over quota (how much is
little?) if this is possible.

> Is it too complicated?
>  

yes.

a simple approach is to update an access db at delivery time, and use
this in check_recipient_access.

another approach is to set two quotas for users
- if the "low" quota threshold is reached, a warning is sent to the user
- if the "next" quota level is reached, the user is blocked, and needs
to contact the postmaster to get unlisted (once he purged his mailbox).

this still allows an overquota, so make sure the "next" level is lower
than a hard quota.

now, if this is a corporate site (in contrast to ISP/MSP), you can make
the problem "social|policy" one instead of a technical one: do not
enforce quotas on the mail server. instead, warn users if they are about
to reach their limit and block them if they don't purge...



Reply | Threaded
Open this post in threaded view
|

Re: RFC: Check mail quota at a mail relay (backscatter)

HAKNER J
In reply to this post by Jozsef Kadlecsik
> Is there a way/would it be feasible to design a Postfix service, which
> could be called to query the next-hop destination *before* accepting the
> mail? It could be a standard SMTP client issuing always
>
> MAIL FROM:<foo@src> SIZE=xxx
> RCPT TO:<bar@dst>

I see what you are saying and I would like to do a very similar thing,
in having flexible quotas and per-user message size limits.  I too would
like to reject the over-quota recipients at the RCPT TO: phase, rather
than accept the message and have to generate a bounce.  Unfortunately,
this requires knowing the message size before RCPT TO:, so the smtpd can
reject specific recipients when the message has more than one recipient
and some recipients are over limit and some are not.  I do not believe that
smtp clients are _required_ to use the SIZE= extension, and so the message
size is not known until after the DATA phase.

Does anyone see it differently?
Reply | Threaded
Open this post in threaded view
|

Re: RFC: Check mail quota at a mail relay (backscatter)

Wietse Venema
In reply to this post by Jozsef Kadlecsik
Jozsef Kadlecsik:

> Hi,
>
> I'd like to prevent to generate bounces at a mail relay due to the
> recipient being over the mailbox size limit or quota at the next-hop
> destination machine.
>
> Is there a way/would it be feasible to design a Postfix service, which
> could be called to query the next-hop destination *before* accepting the
> mail? It could be a standard SMTP client issuing always
>
> MAIL FROM:<foo@src> SIZE=xxx
> RCPT TO:<bar@dst>

Use an access map (updated a few times a day) or policy server (use
a cache for already looked up information).

> Assuming that the queried next-hop destinations run Postfix as well (;-),
> we'd need another service to connect to, which could be a combined simple
> smtpd & enhanced local service answering the question whether the message
> would fit into the mailbox or disk quota of the recipient.

That would require reject_unverified_recipient, with short caching
for positive and negative replies.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: RFC: Check mail quota at a mail relay (backscatter)

Jozsef Kadlecsik
In reply to this post by mouss-2
On Mon, 5 May 2008, mouss wrote:

> you can use a policy service called at recipient and data stages. the
> service must keep "state" between the calls (so as to "link" the
> recipients with the message size). once it knows the size, it can query
> a remote daemon (you can use http or whatever protocol you want) to see
> whether the user is over quota or not.

Yes, it's doable and looks simpler than what I proposed. The only drawback
is that the approach requires an external process at the destinations and
it's harder to get the fellow admins to install such a program than to ask
them to upgrade postfix.
 
> note however that if the recipient has mail still queued on the mail
> relay, your remote query will miss it. other race conditions may happen
> as well. so you should allows for a "little" over quota (how much is
> little?) if this is possible.

That's not a problem, it's not going to be rocket-science ;-).
 
> a simple approach is to update an access db at delivery time, and use
> this in check_recipient_access.

Sorry, I don't understand it. What kind of access db could be updated at
where? At the final delivery, updated by the 'local' process? Something I
must overlook then.
 
> another approach is to set two quotas for users
> - if the "low" quota threshold is reached, a warning is sent to the user
> - if the "next" quota level is reached, the user is blocked, and needs to
> contact the postmaster to get unlisted (once he purged his mailbox).

I don't have any control over the destination machines, just at the relay
farm.

Best regards,
Jozsef
-
E-mail  : [hidden email], [hidden email]
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary
Reply | Threaded
Open this post in threaded view
|

Re: RFC: Check mail quota at a mail relay (backscatter)

Jozsef Kadlecsik
In reply to this post by Wietse Venema
On Mon, 5 May 2008, Wietse Venema wrote:

> > Is there a way/would it be feasible to design a Postfix service, which
> > could be called to query the next-hop destination *before* accepting the
> > mail? It could be a standard SMTP client issuing always
> >
> > MAIL FROM:<foo@src> SIZE=xxx
> > RCPT TO:<bar@dst>
>
> Use an access map (updated a few times a day) or policy server (use
> a cache for already looked up information).

I must have a blind spot, but I don't get the access map idea. What should
update the info (what kind of info?) in the access map? As mouss
suggested, a policy daemon could query the remote destinations about the
quota limits of the user, so I'll look into this possibility.

But isn't there a simple way to get the delivery results out of the smtp
process of Postfix (I mean besides monitoring the log file) and feed it
into a process or database? That way an approximate solution could be set
up: if n unsuccessful delivery attempts detected in a row, then stop
delivering to the recipient (reject) for a couple of hours.

Best regards,
Jozsef
-
E-mail  : [hidden email], [hidden email]
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary
Reply | Threaded
Open this post in threaded view
|

Re: RFC: Check mail quota at a mail relay (backscatter)

mouss-2
In reply to this post by Jozsef Kadlecsik
Jozsef Kadlecsik wrote:

> On Mon, 5 May 2008, mouss wrote:
>  
> [snip]
>> a simple approach is to update an access db at delivery time, and use
>> this in check_recipient_access.
>>    
>
> Sorry, I don't understand it. What kind of access db could be updated at
> where? At the final delivery, updated by the 'local' process? Something I
> must overlook then.
>  
>  
>> another approach is to set two quotas for users
>> - if the "low" quota threshold is reached, a warning is sent to the user
>> - if the "next" quota level is reached, the user is blocked, and needs to
>> contact the postmaster to get unlisted (once he purged his mailbox).
>>    
>
> I don't have any control over the destination machines, just at the relay
> farm.
>  

if you have no control over the remote server, how would you know there
is an over quota event? Even if the remote system is a postfix,
overquota will only be noticed after mail was accepted and queued and is
about to be delivered.

if you can have the final delivery agent to update some quota table,
then you can use this table as an access map in the relay MTA.
otherwise, you can have a cron that periodically checks quotas (only for
recipients that you delivered mail to recently, to avoid scanning a
whole mail store) and updates an access map.




Reply | Threaded
Open this post in threaded view
|

Re: RFC: Check mail quota at a mail relay (backscatter)

Wietse Venema
In reply to this post by Jozsef Kadlecsik
Jozsef Kadlecsik:

> On Mon, 5 May 2008, Wietse Venema wrote:
>
> > > Is there a way/would it be feasible to design a Postfix service, which
> > > could be called to query the next-hop destination *before* accepting the
> > > mail? It could be a standard SMTP client issuing always
> > >
> > > MAIL FROM:<foo@src> SIZE=xxx
> > > RCPT TO:<bar@dst>
> >
> > Use an access map (updated a few times a day) or policy server (use
> > a cache for already looked up information).
>
> I must have a blind spot, but I don't get the access map idea. What should
> update the info (what kind of info?) in the access map? As mouss
> suggested, a policy daemon could query the remote destinations about the
> quota limits of the user, so I'll look into this possibility.

The access map (either flat file or *SQL database) replies with

    554 5.2.2 Mailbox full

when the user's mailbox is full.

> But isn't there a simple way to get the delivery results out of the smtp
> process of Postfix (I mean besides monitoring the log file) and feed it
> into a process or database? That way an approximate solution could be set
> up: if n unsuccessful delivery attempts detected in a row, then stop
> delivering to the recipient (reject) for a couple of hours.

I already mentioned this in the text that you deleted from my reply.

The feature is called reject_unverified_RECIPIENT. The information
comes from the Postfix SMTP client when it talks to the next-hop
host, and the information is stored into $address_verify_map. You
control the process with the address_verify_negative_* and the
unverified_recipient_reject_code parameters.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: RFC: Check mail quota at a mail relay (backscatter)

Jozsef Kadlecsik
On Tue, 6 May 2008, Wietse Venema wrote:

> > But isn't there a simple way to get the delivery results out of the smtp
> > process of Postfix (I mean besides monitoring the log file) and feed it
> > into a process or database? That way an approximate solution could be set
> > up: if n unsuccessful delivery attempts detected in a row, then stop
> > delivering to the recipient (reject) for a couple of hours.
>
> I already mentioned this in the text that you deleted from my reply.
>
> The feature is called reject_unverified_RECIPIENT. The information
> comes from the Postfix SMTP client when it talks to the next-hop
> host, and the information is stored into $address_verify_map. You
> control the process with the address_verify_negative_* and the
> unverified_recipient_reject_code parameters.

Perfect! Thank you, that'll solve the problem. (I have never used address
verification.)

Best regards,
Jozsef
-
E-mail  : [hidden email], [hidden email]
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary