Postfix - Procmail -Cyrus IMAP

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Postfix - Procmail -Cyrus IMAP

Björn Puttmann
Hello List!

I have set up some Mailservers using postfix as MTA, procmail for
filtering (Spam, Virus and especially handling of bounced emails from
newsletters send by these mailservers) and cyrus as imap server.

All in all this stack works brilliant. I'm facing one problem, though.
If a users mailbox is over quota and therefor can not recieve any mails
anymore no notification will be send to the sender of the email.

In the main.cf mailbox_transport is set to procmail.
procmailrc defines some recipes and finally delivers to local users via
/usr/lib/cyrus-imapd/deliver -f $SENDER -a $USER $USER

If the target mailbox is over qouta deliver will return an exitcode of  
75
which in turn can be handled by another recipe, creating a notification
mail for the original sender. This approach is not really satisfactory  
due
to some reasons:

1) most importantly this could end up in emails bouncing back and forth
from one mail account to another if both accounts create a bounce  
message.

2) the return code of 75 is (as far as i know) not really specific. It  
only refers
to a temporary failure while trying to deliver the mail locally.

The maillog gives the following:
Oct 15 13:06:23 postfix lmtpunix[25617]: lmtp connection preauth'd as  
postman
Oct 15 13:06:23 postfix lmtpunix[25617]: verify_user(xxxxxxxx) failed:  
Over quota
Oct 15 13:06:23 postfix postfix/pipe[25677]: C8A5C4B42A:  
to=<user@test>, orig_to=<user@test>, relay=procmail, delay=3.5,  
delays=0.04/0.01/0/3.5, dsn=2.0.0, status=sent (delivered via procmail  
service)

My question is:

Is there any way to prevent postfix from calling mailbox_transport
for an account that is over quota and instead produce a bounce message
notifying the sender of the fact that the mail could not be deliverd?

If not, is there way to make sure that procmail will not end up bouncing
messages back and forth if both accounts produce bounces?

Thanks for any input and take care,
Bjoern
Reply | Threaded
Open this post in threaded view
|

Re: Postfix - Procmail -Cyrus IMAP

Barney Desmond
My experience with quotas is quite limited, but...

Björn Puttmann wrote:
> 1) most importantly this could end up in emails bouncing back and forth
> from one mail account to another if both accounts create a bounce message.

If your recipe is generating these mails, it should add a header which
it can check for, eg. "X-Loop-Prevention" or something. In the worst
case scenario, you send an "over quota" message, the other side returns
one to you, then you stop when you see your own header.

> 2) the return code of 75 is (as far as i know) not really specific. It
> only refers
> to a temporary failure while trying to deliver the mail locally.

Consider instead that is *is* a temporary failure. The user can't accept
mail right now, but they will later when/if they clean out their inbox.
I know this isn't a solution, but it's worth considering why this is a
temporary failure instead of a permanent one. Returning an message to
the sender like this is like a bounce, is there a way to tell `deliver`
to treat it like one and give a better return code?

> Is there any way to prevent postfix from calling mailbox_transport
> for an account that is over quota and instead produce a bounce message
> notifying the sender of the fact that the mail could not be deliverd?

My knowledge is shaky here, but this would require the delivery agent
passing a message back to postfix to tell it it's no-good. I assume this
isn't an issue for postfix's inbuilt local delivery agent, as it just
tries to write files and will know immediately if there's a problem.
There's no way to know that delivery will fail unless you try, and
postfix doesn't know about quotas.


signature.asc (258 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Postfix - Procmail -Cyrus IMAP

Björn Puttmann

Hopefully this mail will not be empty as well...
It seems MTA do not like me these days ;)

My experience with quotas is quite limited, but...

Björn Puttmann wrote:
1) most importantly this could end up in emails bouncing back and forth
from one mail account to another if both accounts create a bounce message.

If your recipe is generating these mails, it should add a header which
it can check for, eg. "X-Loop-Prevention" or something. In the worst
case scenario, you send an "over quota" message, the other side returns
one to you, then you stop when you see your own header.

Ah yes, after writing the first post and getting meself a coffee I thought about
something similar ;) It's always a good idea to explain ones problem to 
someone else - might end up with a new idea...

Searching the web for your suggestion came up with some good starters

for the interested here my procmailrc

LOGFILE="/var/log/procmail"
VERBOSE=on

CHECKSUM=/usr/bin/md5sum
DQ='"'
DELIVERMAIL="/usr/lib/cyrus-imapd/deliver"
MAILDOMAIN="your-domain.tld"
DAEMON="mailer-daemon@$MAILDOMAIN"
SENDMAIL="/usr/sbin/sendmail -i -f \
  $DAEMON -t $SENDER" 

# Mailbox-Delivery
INBOX="$DELIVERMAIL -f $SENDER -a $USER $USER"

# enable / disable spam && virus checks
CHECK = 0

### Spamassassin && ClamAV
# Check
:0
* $CHECK ?? 1
{
    :0fw
    * < 110000
    | /usr/bin/spamc -u filter

    # virus scan only if the mail is not classified as spam
    :0
    * ^X-Spam-Status: No
    {
        :0fw
        | /usr/local/bin/clamassassin
    }
}

# move infected mails to /dev/null
:0 w
* ^X-Virus-Status: Yes
/dev/null

# deliver to INBOX
:0 w
| $INBOX

# if this went wrong
:0 e
{
    EXITCODE=$?
    # temporary failure?
    :0 w
    * EXITCODE ?? [75]
    {
        EXITCODE=0
       
# prevent mail loops
:0
        * ^X-Loop: $DAEMON
        /dev/null

:0
BOUNCEPID=`echo $$`
                LOCALRECIPIENT=`formail  -t -xTo:`
                ORIGINALSENDER=`formail -rt -xTo:`
                LOCALDOMAINMX=`dig MX +short ${MAILDOMAIN}|sed -e 's/. / /g' -e 's/.$//g' |awk '{print $1}'`
                DATE=`date`    
                MIMEBREAK="`cat bouncetemp.${BOUNCEPID} | ${CHECKSUM} | sed -e 's/  -//'`.`date '+%s'`/${HOST}.${MAILDOMAIN}"
}

:0 c:
bouncetemp.${BOUNCEPID}

:0
| ( formail -rt \
-I"From: $DAEMON (Mail Delivery Subsystem)" \
-I"Subject: Returned mail: see transcript for details" \
-I"Auto-Submitted: auto-generated (failure)" \
-I"MIME-Version: 1.0" \
-I"Content-Type: multipart/report; report-type=delivery-status; boundary=${DQ}${MIMEBREAK}${DQ}" \
-A"X-Loop: ${DAEMON}";\
printf "\nThis is a MIME-encapsulated message\n";\
printf "\n--%s\n\n" "${MIMEBREAK}";\
printf "The original message was received at %s" "${DATE}";\
printf " from %s\n" "${ORIGINALSENDER}";\
printf "\n----- The original mail could not be deliverd to the following addresses due to a temporary error -----";\
printf "\n\n451 Temporary local deliveriy problem for %s - please try again later... \n" "${LOCALRECIPIENT}";\
printf "\n--%s\nContent-Type: message/delivery-status\n\n" "${MIMEBREAK}";\
printf "Reporting-MTA: dns; %s\n" "postfix@${MAILDOMAIN}";\
printf "Arrival-Date: %s\n\n" "${DATE}";\
printf "Final-Recipient: RFC822; %s\n" "${LOCALRECIPIENT}";\
printf "Last-Attempt-Date: %s\n" "`date`";\
printf "\n--%s\nContent-Type: message/rfc822\n\n" "${MIMEBREAK}";\
cat bouncetemp.${BOUNCEPID};\
printf "\n--%s--\n\n" "${MIMEBREAK}";\
rm -f bouncetemp.${BOUNCEPID} \
) | ${SENDMAIL} -od -t
    }

An nice to have would be a reverse lookup of the original sender. Thus implementing a basic 
filter for not sending bounces to spam addresses.

2) the return code of 75 is (as far as i know) not really specific. It
only refers
to a temporary failure while trying to deliver the mail locally.

Consider instead that is *is* a temporary failure. The user can't accept
mail right now, but they will later when/if they clean out their inbox.
I know this isn't a solution, but it's worth considering why this is a
temporary failure instead of a permanent one. Returning an message to
the sender like this is like a bounce, is there a way to tell `deliver`
to treat it like one and give a better return code?

Yes, I think you are right. The original sender really only needs to know
that his mail did not reach the recipient. Everything else is not really 
neccessary...

Thanks for your reply and all the best,
Bjoern
-- 
dbap GmbH - one-stop e-business
phone +49 251 609979-0 / fax +49 251 609979-99
Martin-Luther-King Weg 20 - 48155 Muenster, Germany
http://www.dbap.de

Die dbap GmbH ist Full-Service Dienstleister für mittelständische
Unternehmen. Wir betreuen die Einbindung modernster E-Business-
Technologien in die bestehenden Geschäftsprozesse auf Basis der
hauseigenen Softwareplattform dStore SmartSuite.

dbap GmbH, Sitz: Münster HRB 5891, Amtsgericht Münster
Geschäftsführer: Björn Dolkemeier, Stefan Goebel 

Reply | Threaded
Open this post in threaded view
|

Re: Postfix - Procmail -Cyrus IMAP

mouss-2
In reply to this post by Björn Puttmann
Björn Puttmann a écrit :

> Hello List!
>
> I have set up some Mailservers using postfix as MTA, procmail for
> filtering (Spam, Virus and especially handling of bounced emails from
> newsletters send by these mailservers) and cyrus as imap server.
>
> All in all this stack works brilliant. I'm facing one problem, though.
> If a users mailbox is over quota and therefor can not recieve any mails
> anymore no notification will be send to the sender of the email.
>
> In the main.cf mailbox_transport is set to procmail.
> procmailrc defines some recipes and finally delivers to local users via
> /usr/lib/cyrus-imapd/deliver -f $SENDER -a $USER $USER
>
> If the target mailbox is over qouta deliver will return an exitcode of 75
> which in turn can be handled by another recipe, creating a notification
> mail for the original sender. This approach is not really satisfactory due
> to some reasons:
>
> 1) most importantly this could end up in emails bouncing back and forth
> from one mail account to another if both accounts create a bounce message.

That's why the null sender was invented years ago :)

when sending bounces, set the envelope sender to: <>.

PS. I don't like the X-Loop header hack. the other side can remove it.
The null sender is standard and robust.

>
> 2) the return code of 75 is (as far as i know) not really specific. It
> only refers
> to a temporary failure while trying to deliver the mail locally.
>
> The maillog gives the following:
> Oct 15 13:06:23 postfix lmtpunix[25617]: lmtp connection preauth'd as
> postman
> Oct 15 13:06:23 postfix lmtpunix[25617]: verify_user(xxxxxxxx) failed:
> Over quota
> Oct 15 13:06:23 postfix postfix/pipe[25677]: C8A5C4B42A: to=<user@test>,
> orig_to=<user@test>, relay=procmail, delay=3.5, delays=0.04/0.01/0/3.5,
> dsn=2.0.0, status=sent (delivered via procmail service)

so your pocmail doesn't pass the error back to postfix. This means mail
is lost (unless you implement a queue manager inside procmail, but then
you can also implement an smtp server and drop postfix :).


>
> My question is:
>
> Is there any way to prevent postfix from calling mailbox_transport
> for an account that is over quota and instead produce a bounce message
> notifying the sender of the fact that the mail could not be deliverd?
>

use check_recipient_access with a map that you manage via your procmail
or anything else. This way postfix will reject (not bounce. bounces are
bad. only bounce if you don't have a choice).

> If not, is there way to make sure that procmail will not end up bouncing
> messages back and forth if both accounts produce bounces?
>
> Thanks for any input and take care,
> Bjoern

Reply | Threaded
Open this post in threaded view
|

Re: Postfix - Procmail -Cyrus IMAP

Reinaldo Gil Lima de Carvalho
In reply to this post by Björn Puttmann
>
> Is there any way to prevent postfix from calling mailbox_transport
> for an account that is over quota and instead produce a bounce message
> notifying the sender of the fact that the mail could not be deliverd?
>
> If not, is there way to make sure that procmail will not end up bouncing
> messages back and forth if both accounts produce bounces?
>
> Thanks for any input and take care,
> Bjoern
>

I wrote a polycd-daemon (python) to check quota on cyrus using imap
protocol (as cyrus admin user).

This policy-daemon search on ldap to check if email is a alias, to get
and retrive quota from real user.

This program is not complete to public distribution, I wrote on
half-hour to a customer. Many options as hard-coded and you need read
and understand the code to be usefull for you.

http://www.nautilus.com.br/~rei/material/lost+found/policyd-quota

--
Reinaldo de Carvalho
http://korreio.sf.net
http://python-cyrus.sf.net
Reply | Threaded
Open this post in threaded view
|

Re: Postfix - Procmail -Cyrus IMAP

Björn Puttmann

Am 17.10.2008 um 14:28 schrieb Reinaldo de Carvalho:

> I wrote a polycd-daemon (python) to check quota on cyrus using imap
> protocol (as cyrus admin user).
>
> This policy-daemon search on ldap to check if email is a alias, to get
> and retrive quota from real user.
>
> This program is not complete to public distribution, I wrote on
> half-hour to a customer. Many options as hard-coded and you need read
> and understand the code to be usefull for you.
>
> http://www.nautilus.com.br/~rei/material/lost+found/policyd-quota


Hello Reinaldo,

thanks for the link. I will have a look at it, though ruby would be more
up my alley ;)

Here is the final version of my procmailrc file, maybe it could be  
useful
for somebody else as well....

LOGFILE="/var/log/procmail"
VERBOSE=no

# command definitions
CHECKSUM=/usr/bin/md5sum
DQ='"'
DELIVERMAIL="/usr/lib/cyrus-imapd/deliver"
MAILDOMAIN="dbap.de"
DAEMON="mailer-daemon@$MAILDOMAIN"
SENDMAIL="/usr/sbin/sendmail -i -f $DAEMON -t $SENDER"
INBOX="$DELIVERMAIL -f $SENDER -a $USER $USER"

# initalize vars
NL = "
"
LOCALRECIPIENT=`formail  -t -xTo:`
ORIGINALSENDER=`formail -rt -xTo:`
LOG=`date`": Processing mail from ${ORIGINALSENDER} to $
{LOCALRECIPIENT}${NL}"

# enable / disable spam && virus check
CHECK = 0

:0
* $CHECK ?? 0
{
   LOG=`date`": Spam and Viruschecks are disabled.${NL}"
}

# Spamassassin && ClamAV
:0
* $CHECK ?? 1
{
        LOG=`date`": Starting spam check... ${NL}"
      :0fw
      * < 110000
      | /usr/bin/spamc -u filter

        LOG=`date`": Starting virus check... ${NL}"
         :0fw
         | /usr/local/bin/clamassassin

        :0
        * ^X-Virus-Status: Yes
        {
                LOG=`date`": The message from ${ORIGINALSENDER} to ${LOCALRECIPIENT}  
contained a virus and was deleted.${NL}"
                :0
                /dev/null
        }
}

# deliver to INBOX
:0 w
| $INBOX

# if this went wrong
:0 e
{
      EXITCODE=$?

        # drop all non temporary errors (e.g. Mailbox does not exist etc.)
        :0
        * ! EXITCODE ?? 75
        /dev/null

        # temporary failure?
    :0 w
      * EXITCODE ?? 75
      {
          EXITCODE=0

                :0
          * ^X-Loop: $DAEMON
          /dev/null

                :0
                {
                        BOUNCEPID=`echo $$`
                  LOCALDOMAINMX=`dig MX +short ${MAILDOMAIN}|sed -e  
's/. / /g' -e 's/.$//g' |awk '{print $1}'`
                  DATE=`date`
                  MIMEBREAK="`cat bouncetemp.${BOUNCEPID} | $
{CHECKSUM} | sed -e 's/  -//'`.`date '+%s'`/${HOST}.${MAILDOMAIN}"
                        LOG=`date`": local delivery to ${LOCALRECIPIENT} for this message  
had a temporary error. Bounce message to ${ORIGINALSENDER} will be  
send.${NL}"
                }

                # if the original mail was marked as spam do not send bounce message
                :0
                * ^X-Spam-Status: Yes
                {
                        LOG=`date`": sending of bounce message to ${ORIGINALSENDER} was  
skipped because the original message was marked as spam.${NL}"
                        :0
                        /dev/null
                }

                :0 c:
                bouncetemp.${BOUNCEPID}

                :0
                | ( formail -rt \
                        -I"From: $DAEMON (Mail Delivery Subsystem)" \
                        -I"Subject: Returned mail: see transcript for details" \
                        -I"Auto-Submitted: auto-generated (failure)" \
                        -I"MIME-Version: 1.0" \
                        -I"Errors-To: <>" \
                        -I"Content-Type: multipart/report; report-type=delivery-status;  
boundary=${DQ}${MIMEBREAK}${DQ}" \
                        -A"X-Loop: ${DAEMON}";\
                  printf "\nThis is a MIME-encapsulated message\n";\
                  printf "\n--%s\n\n" "${MIMEBREAK}";\
                  printf "The original message was received at %s" "${DATE}";\
                  printf " from %s\n" "${ORIGINALSENDER}";\
                  printf "\n----- The original mail could not be deliverd to the  
following addresses due to a temporary error -----";\
                  printf "\n\n451 Temporary local deliveriy problem for %s - please  
try again later... \n" "${LOCALRECIPIENT}";\
                  printf "\n--%s\nContent-Type: message/delivery-status\n\n" "$
{MIMEBREAK}";\
                  printf "Reporting-MTA: dns; %s\n" "postfix@${MAILDOMAIN}";\
                  printf "Arrival-Date: %s\n\n" "${DATE}";\
                  printf "Final-Recipient: RFC822; %s\n" "${LOCALRECIPIENT}";\
                  printf "Last-Attempt-Date: %s\n" "`date`";\
                  printf "\n--%s\nContent-Type: message/rfc822\n\n" "${MIMEBREAK}";\
                  cat bouncetemp.${BOUNCEPID};\
                  printf "\n--%s--\n\n" "${MIMEBREAK}";\
                  rm -f bouncetemp.${BOUNCEPID} \
                ) | ${SENDMAIL} -od -t
      }
}

--
dbap GmbH - one-stop e-business
phone +49 251 609979-0 / fax +49 251 609979-99
Martin-Luther-King Weg 20 - 48155 Muenster, Germany
http://www.dbap.de

Die dbap GmbH ist Full-Service Dienstleister für mittelständische
Unternehmen. Wir betreuen die Einbindung modernster E-Business-
Technologien in die bestehenden Geschäftsprozesse auf Basis der
hauseigenen Softwareplattform dStore SmartSuite.

dbap GmbH, Sitz: Münster HRB 5891, Amtsgericht Münster
Geschäftsführer: Björn Dolkemeier, Stefan Goebel