Architecture design for frontend postfix server

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Architecture design for frontend postfix server

bsd@todoo.biz
Hello folks,


I am trying to figure out what is the best solution in order to filter incoming e-mail on a front-end relay server quite heavily loaded (100.000 messages / day).


We are using LDAP querry in amavisd-new in order to check if the e-mail account exists in our LDAP DB.


Internet <--> Relay [Postfix + Amavisd-new] <--> Internal postfix
                        | |
                        | |
                        --------[LDAP database]----------



Since update of our amavisd-new server to version 2.7.0 we are having problem with the LDAP lookup.

I was wondering if It was not better to do this lookup before delivering the e-mail to the amavisd process ?

If the answer is positive, should I use the "local_recipient_maps" parameter or is there another more efficient method to be used  ?

E-mail are flowing both from the Internet to the Internal server and from Internal server to the Internet.



Thanks for your support.

––––––––––––––––––––––––––––––––––––––––––––––
---------> Grégory Bernard Director <---------
---------------> www.osnet.eu <---------------
--> Your provider of OpenSource appliances <--
––––––––––––––––––––––––––––––––––––––––––––––
OSnetOSnetOSnetOSnetOSnetOSnetOSnetOSnetOSnetO

Reply | Threaded
Open this post in threaded view
|

Re: Architecture design for frontend postfix server

Patrick Ben Koetter
* bsd <[hidden email]>:
> I am trying to figure out what is the best solution in order to filter
> incoming e-mail on a front-end relay server quite heavily loaded (100.000
> messages / day).

Even if you assume that all those messages will be sent within 8 hours
(business time) it will boil down to ~ 3.5 msg/sec. Server hardware can do
that easily.


> We are using LDAP querry in amavisd-new in order to check if the e-mail
> account exists in our LDAP DB.

Bad idea.

> Internet <--> Relay [Postfix + Amavisd-new] <--> Internal postfix
> | |
> | |
> --------[LDAP database]----------
>
>
>
> Since update of our amavisd-new server to version 2.7.0 we are having
> problem with the LDAP lookup.
>
> I was wondering if It was not better to do this lookup before delivering the
> e-mail to the amavisd process ?

Good idea. Reject any message that can't be delivered immediately. That's
cheap. It takes place in the SMTP session before the payload has been sent and
before a content filter, such as amavis, does ressource intensive filtering.

> If the answer is positive, should I use the "local_recipient_maps" parameter
> or is there another more efficient method to be used  ?

Depends on the namespace the recpient domain is in.

p@rick

--
All technical questions asked privately will be automatically answered on the
list and archived for public access unless privacy is explicitely required and
justified.

saslfinger (debugging SMTP AUTH):
<http://postfix.state-of-mind.de/patrick.koetter/saslfinger/>
Reply | Threaded
Open this post in threaded view
|

Re: Architecture design for frontend postfix server

postfix-5
Hi
we use here since ages
virtual_mailbox_domains = proxy:ldap:/etc/postfix/ldap-domain.cf
and if that is positive (i.e. the domain is local)
virtual_alias_maps = proxy:ldap:/etc/postfix/ldap-alias.cf
with
virtual_transport = virtual

suomi


On 2011-11-09 11:43, Patrick Ben Koetter wrote:

> * bsd<[hidden email]>:
>> I am trying to figure out what is the best solution in order to filter
>> incoming e-mail on a front-end relay server quite heavily loaded (100.000
>> messages / day).
>
> Even if you assume that all those messages will be sent within 8 hours
> (business time) it will boil down to ~ 3.5 msg/sec. Server hardware can do
> that easily.
>
>
>> We are using LDAP querry in amavisd-new in order to check if the e-mail
>> account exists in our LDAP DB.
>
> Bad idea.
>
>> Internet<-->  Relay [Postfix + Amavisd-new]<-->  Internal postfix
>> | |
>> | |
>> --------[LDAP database]----------
>>
>>
>>
>> Since update of our amavisd-new server to version 2.7.0 we are having
>> problem with the LDAP lookup.
>>
>> I was wondering if It was not better to do this lookup before delivering the
>> e-mail to the amavisd process ?
>
> Good idea. Reject any message that can't be delivered immediately. That's
> cheap. It takes place in the SMTP session before the payload has been sent and
> before a content filter, such as amavis, does ressource intensive filtering.
>
>> If the answer is positive, should I use the "local_recipient_maps" parameter
>> or is there another more efficient method to be used  ?
>
> Depends on the namespace the recpient domain is in.
>
> p@rick
>
Reply | Threaded
Open this post in threaded view
|

Re: Architecture design for frontend postfix server

Nerijus Kislauskas
In reply to this post by bsd@todoo.biz
On 11/09/2011 12:09 PM, bsd wrote:
> If the answer is positive, should I use the "local_recipient_maps" parameter or is there another more efficient method to be used  ?

I guess the answer is "relay_recipient_maps", because your first postfix
instance is a SMTP relay server.

Indeed, we have a problem with LDAP lookups too, but dog's voice doesn't
go to heaven. :)
--
Pagarbiai,
Nerijus Kislauskas
KTU ITPI, Litnet valdymo centras
Studentu g. 48a - 101, Kaunas
tel.: (8~37) 30 06 45
mob. tel.: 8-614-93889
e-mail.: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Architecture design for frontend postfix server

bsd@todoo.biz
In reply to this post by Patrick Ben Koetter

Le 9 nov. 2011 à 11:43, Patrick Ben Koetter a écrit :

> * bsd <[hidden email]>:
>> I am trying to figure out what is the best solution in order to filter
>> incoming e-mail on a front-end relay server quite heavily loaded (100.000
>> messages / day).
>
> Even if you assume that all those messages will be sent within 8 hours
> (business time) it will boil down to ~ 3.5 msg/sec. Server hardware can do
> that easily.
>
>
>> We are using LDAP querry in amavisd-new in order to check if the e-mail
>> account exists in our LDAP DB.
>
> Bad idea.

That was also my feeling about this…

>
>> Internet <--> Relay [Postfix + Amavisd-new] <--> Internal postfix
>> | |
>> | |
>> --------[LDAP database]----------
>>
>>
>>
>> Since update of our amavisd-new server to version 2.7.0 we are having
>> problem with the LDAP lookup.
>>
>> I was wondering if It was not better to do this lookup before delivering the
>> e-mail to the amavisd process ?
>
> Good idea. Reject any message that can't be delivered immediately. That's
> cheap. It takes place in the SMTP session before the payload has been sent and
> before a content filter, such as amavis, does ressource intensive filtering.

Shall I do that using the "Postfix Before-Queue Content Filter", if so is It feasible or do you advise me to do that in the normal SMTP server using the classic "virtual_alias_maps"


That would give smthg like :



Unfiltered -> before Q   -> smtpd -> cleanup -> postfix Q -> smtpd (filtered)
                                 ^      |
                                 |      v
                           smtpd 10026     smtp
                                 ^      |
                                 |      v
                              Amavisd filter 10025









>
>> If the answer is positive, should I use the "local_recipient_maps" parameter
>> or is there another more efficient method to be used  ?
>
> Depends on the namespace the recpient domain is in.

recipient would be in smthg like

user@mydomain(s)

where domains = less than 10 domains.


>
> p@rick
>
> --
> All technical questions asked privately will be automatically answered on the
> list and archived for public access unless privacy is explicitely required and
> justified.
>
> saslfinger (debugging SMTP AUTH):
> <http://postfix.state-of-mind.de/patrick.koetter/saslfinger/>


––––––––––––––––––––––––––––––––––––––––––––––
---------> Grégory Bernard Director <---------
---------------> www.osnet.eu <---------------
--> Your provider of OpenSource appliances <--
––––––––––––––––––––––––––––––––––––––––––––––
OSnetOSnetOSnetOSnetOSnetOSnetOSnetOSnetOSnetO

Reply | Threaded
Open this post in threaded view
|

Re: Architecture design for frontend postfix server

Patrick Ben Koetter
* bsd <[hidden email]>:
> > Good idea. Reject any message that can't be delivered immediately. That's
> > cheap. It takes place in the SMTP session before the payload has been sent and
> > before a content filter, such as amavis, does ressource intensive filtering.
>
> Shall I do that using the "Postfix Before-Queue Content Filter", if so is It
> feasible or do you advise me to do that in the normal SMTP server using the
> classic "virtual_alias_maps"


Before or After Queue Filtering depends on legal requirements and how your
server can take the load.

In Germany, for example, you have to reject in session or you will get intro
trouble if you decide at a later stage to accept, but discard the message
later, because to the law this means you suppress delivery - which is a crime
in Germany.

And, if your server is so busy that it can't process message filtering while
the client waits for the server to accept the message, you should consider
After Queue Filtering.

As for your question regarding *_maps. Is your server a border filter (vulgo
SMTP Gateway) which accepts messages and forwards them to other servers or is
it a final destination where clients come to pick up messages?

p@rick


--
All technical questions asked privately will be automatically answered on the
list and archived for public access unless privacy is explicitely required and
justified.

saslfinger (debugging SMTP AUTH):
<http://postfix.state-of-mind.de/patrick.koetter/saslfinger/>
Reply | Threaded
Open this post in threaded view
|

Re: Architecture design for frontend postfix server

bsd@todoo.biz

Le 9 nov. 2011 à 14:24, Patrick Ben Koetter a écrit :

> * bsd <[hidden email]>:
>>> Good idea. Reject any message that can't be delivered immediately. That's
>>> cheap. It takes place in the SMTP session before the payload has been sent and
>>> before a content filter, such as amavis, does ressource intensive filtering.
>>
>> Shall I do that using the "Postfix Before-Queue Content Filter", if so is It
>> feasible or do you advise me to do that in the normal SMTP server using the
>> classic "virtual_alias_maps"
>
>
> Before or After Queue Filtering depends on legal requirements and how your
> server can take the load.
>
> In Germany, for example, you have to reject in session or you will get intro
> trouble if you decide at a later stage to accept, but discard the message
> later, because to the law this means you suppress delivery - which is a crime
> in Germany.
>
> And, if your server is so busy that it can't process message filtering while
> the client waits for the server to accept the message, you should consider
> After Queue Filtering.
>
> As for your question regarding *_maps. Is your server a border filter (vulgo
> SMTP Gateway) which accepts messages and forwards them to other servers or is
> it a final destination where clients come to pick up messages?

It is a SMTP Gateway delivering to another internal server.
In fact there are two Gateways server with equal MX weight.


>
> p@rick
>
>
> --
> All technical questions asked privately will be automatically answered on the
> list and archived for public access unless privacy is explicitely required and
> justified.
>
> saslfinger (debugging SMTP AUTH):
> <http://postfix.state-of-mind.de/patrick.koetter/saslfinger/>


––––––––––––––––––––––––––––––––––––––––––––––
---------> Grégory Bernard Director <---------
---------------> www.osnet.eu <---------------
--> Your provider of OpenSource appliances <--
––––––––––––––––––––––––––––––––––––––––––––––
OSnetOSnetOSnetOSnetOSnetOSnetOSnetOSnetOSnetO

Reply | Threaded
Open this post in threaded view
|

Re: Architecture design for frontend postfix server

Patrick Ben Koetter
* bsd <[hidden email]>:

>
> Le 9 nov. 2011 à 14:24, Patrick Ben Koetter a écrit :
>
> > * bsd <[hidden email]>:
> >>> Good idea. Reject any message that can't be delivered immediately. That's
> >>> cheap. It takes place in the SMTP session before the payload has been sent and
> >>> before a content filter, such as amavis, does ressource intensive filtering.
> >>
> >> Shall I do that using the "Postfix Before-Queue Content Filter", if so is It
> >> feasible or do you advise me to do that in the normal SMTP server using the
> >> classic "virtual_alias_maps"
> >
> >
> > Before or After Queue Filtering depends on legal requirements and how your
> > server can take the load.
> >
> > In Germany, for example, you have to reject in session or you will get intro
> > trouble if you decide at a later stage to accept, but discard the message
> > later, because to the law this means you suppress delivery - which is a crime
> > in Germany.
> >
> > And, if your server is so busy that it can't process message filtering while
> > the client waits for the server to accept the message, you should consider
> > After Queue Filtering.
> >
> > As for your question regarding *_maps. Is your server a border filter (vulgo
> > SMTP Gateway) which accepts messages and forwards them to other servers or is
> > it a final destination where clients come to pick up messages?
>
> It is a SMTP Gateway delivering to another internal server.
> In fact there are two Gateways server with equal MX weight.

Read into relay_domains and relay_recpient_maps:
<http://www.postfix.org/STANDARD_CONFIGURATION_README.html#firewall>

p@rick

--
All technical questions asked privately will be automatically answered on the
list and archived for public access unless privacy is explicitely required and
justified.

saslfinger (debugging SMTP AUTH):
<http://postfix.state-of-mind.de/patrick.koetter/saslfinger/>