load balancing among mail servers

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

load balancing among mail servers

angela vulpio
Hi,
I need to create an infrastructure that allows to divide a list of mails to send among a series of postfix mail servers.

For example, I imagine this situation so :
I install a series of postfix mail servers and when it's necessary to send a mail it's enough to send it to an IP address that  the actual delivery of the mail is delegated to a server that has the lowest charge of jobs...

I'm wondering if a particular configuration of postfix can be useful to resolve this issue...

Someone advised me to insert in the DNS zone a list of MX records defined with the same level of priority so the DNS server will choose one of them without invoking always the same mail server....
It could be an idea, in my opinion, but I'd prefer a "less  random" solution and a more scientific one....

Thanks for every suggestion you'll send to me....
Reply | Threaded
Open this post in threaded view
|

Re: load balancing among mail servers

Eero Volotinen-2
2010/2/16 aa <[hidden email]>:

> Hi,
> I need to create an infrastructure that allows to divide a list of mails to
> send among a series of postfix mail servers.
>
> For example, I imagine this situation so :
> I install a series of postfix mail servers and when it's necessary to send a
> mail it's enough to send it to an IP address that  the actual delivery of
> the mail is delegated to a server that has the lowest charge of jobs...
>
> I'm wondering if a particular configuration of postfix can be useful to
> resolve this issue...
>
> Someone advised me to insert in the DNS zone a list of MX records defined
> with the same level of priority so the DNS server will choose one of them
> without invoking always the same mail server....
> It could be an idea, in my opinion, but I'd prefer a "less  random" solution
> and a more scientific one....
>
> Thanks for every suggestion you'll send to me....
>

How about OpenBSD relayd: https://calomel.org/relayd.html

--
Eero
Reply | Threaded
Open this post in threaded view
|

Re: load balancing among mail servers

Patrick Chemla
In reply to this post by angela vulpio
Le 16/02/2010 15:09, aa a écrit :
>
> Someone advised me to insert in the DNS zone a list of MX records
> defined with the same level of priority so the DNS server will choose
> one of them without invoking always the same mail server....
> It could be an idea, in my opinion, but I'd prefer a "less  random"
> solution and a more scientific one....

I did this to load balanced among 40 mail servers. It works very very
fine with a load difference of less than 2% among the 40 servers. I
don't think it is too random.



Patrick

Reply | Threaded
Open this post in threaded view
|

Re: load balancing among mail servers

jeffrey j donovan
In reply to this post by angela vulpio

On Feb 16, 2010, at 8:09 AM, aa wrote:

> Someone advised me to insert in the DNS zone a list of MX records  
> defined with the same level of priority so the DNS server will  
> choose one of them without invoking always the same mail server....
> It could be an idea, in my opinion, but I'd prefer a "less  random"  
> solution and a more scientific one....

very easy for smtp relays.

smtp1
smtp2


create a dns name smtp, and your system will round robin query for the  
next available server.
Reply | Threaded
Open this post in threaded view
|

Re: load balancing among mail servers

Massimo Nuvoli
donovan jeffrey j ha scritto:

>
> On Feb 16, 2010, at 8:09 AM, aa wrote:
>
>> Someone advised me to insert in the DNS zone a list of MX records
>> defined with the same level of priority so the DNS server will choose
>> one of them without invoking always the same mail server....
>> It could be an idea, in my opinion, but I'd prefer a "less  random"
>> solution and a more scientific one....
> very easy for smtp relays.
> smtp1
> smtp2
> create a dns name smtp, and your system will round robin query for the
> next available server.
DNS round robin is bad, it works but is defective for real load
balancing. The client choose the IP to use, this is "random", and
after can use the same ip for a while... this is not random.

The real solution is lvs or keepalived, the choice of the node is done
by the load balancer...

Bye.

massimo.vcf (448 bytes) Download Attachment
signature.asc (258 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: load balancing among mail servers

Banyan He-2
In reply to this post by angela vulpio
Depends on how many public IP addresses you have. I'd like recommend you
to have a try with keepalived. It's the balancing service software. Open
source. The front-end service keepalived will handle TCP request and
forward to the back-end servers you have. You can also setup the filter
also. Pretty nice.

On 2/16/2010 9:09 PM, aa wrote:

> Hi,
> I need to create an infrastructure that allows to divide a list of
> mails to send among a series of postfix mail servers.
>
> For example, I imagine this situation so :
> I install a series of postfix mail servers and when it's necessary to
> send a mail it's enough to send it to an IP address that  the actual
> delivery of the mail is delegated to a server that has the lowest
> charge of jobs...
>
> I'm wondering if a particular configuration of postfix can be useful
> to resolve this issue...
>
> Someone advised me to insert in the DNS zone a list of MX records
> defined with the same level of priority so the DNS server will choose
> one of them without invoking always the same mail server....
> It could be an idea, in my opinion, but I'd prefer a "less  random"
> solution and a more scientific one....
>
> Thanks for every suggestion you'll send to me....

--
Banyan He
Information Security&  Infranstructure Technology
Blog: http://www.rootong.com
Email: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: load balancing among mail servers

jeffrey j donovan
In reply to this post by Massimo Nuvoli

On Feb 16, 2010, at 10:39 AM, Massimo Nuvoli wrote:

> donovan jeffrey j ha scritto:
>>
>> On Feb 16, 2010, at 8:09 AM, aa wrote:
>>
>>> Someone advised me to insert in the DNS zone a list of MX records
>>> defined with the same level of priority so the DNS server will  
>>> choose
>>> one of them without invoking always the same mail server....
>>> It could be an idea, in my opinion, but I'd prefer a "less  random"
>>> solution and a more scientific one....
>> very easy for smtp relays.
>> smtp1
>> smtp2
>> create a dns name smtp, and your system will round robin query for  
>> the
>> next available server.
>
> DNS round robin is bad, it works but is defective for real load
> balancing. The client choose the IP to use, this is "random", and
> after can use the same ip for a while... this is not random.
>
> The real solution is lvs or keepalived, the choice of the node is done
> by the load balancer...
>
> Bye.
> <massimo.vcf>

I wouldn't  say it's bad, it's just cheap. Of course a load balancing  
switch can be purchased $
-j
Reply | Threaded
Open this post in threaded view
|

Re: load balancing among mail servers

Eero Volotinen-2
2010/2/16 donovan jeffrey j <[hidden email]>:

>
> On Feb 16, 2010, at 10:39 AM, Massimo Nuvoli wrote:
>
>> donovan jeffrey j ha scritto:
>>>
>>> On Feb 16, 2010, at 8:09 AM, aa wrote:
>>>
>>>> Someone advised me to insert in the DNS zone a list of MX records
>>>> defined with the same level of priority so the DNS server will choose
>>>> one of them without invoking always the same mail server....
>>>> It could be an idea, in my opinion, but I'd prefer a "less  random"
>>>> solution and a more scientific one....
>>>
>>> very easy for smtp relays.
>>> smtp1
>>> smtp2
>>> create a dns name smtp, and your system will round robin query for the
>>> next available server.
>>
>> DNS round robin is bad, it works but is defective for real load
>> balancing. The client choose the IP to use, this is "random", and
>> after can use the same ip for a while... this is not random.
>>
>> The real solution is lvs or keepalived, the choice of the node is done
>> by the load balancer...
>>
>> Bye.
>> <massimo.vcf>
>
> I wouldn't  say it's bad, it's just cheap. Of course a load balancing switch
> can be purchased $


Well, It is bad because server (client) can cache ip address for long
time and then one of your smtp server fails. It can take long time
before server gets ip address of working smtp server..

--
Eero
Reply | Threaded
Open this post in threaded view
|

Re: load balancing among mail servers

Wietse Venema
In reply to this post by Massimo Nuvoli
Massimo Nuvoli:

> donovan jeffrey j ha scritto:
> >
> > On Feb 16, 2010, at 8:09 AM, aa wrote:
> >
> >> Someone advised me to insert in the DNS zone a list of MX records
> >> defined with the same level of priority so the DNS server will choose
> >> one of them without invoking always the same mail server....
> >> It could be an idea, in my opinion, but I'd prefer a "less  random"
> >> solution and a more scientific one....
> > very easy for smtp relays.
> > smtp1
> > smtp2
> > create a dns name smtp, and your system will round robin query for the
> > next available server.
>
> DNS round robin is bad, it works but is defective for real load
> balancing. The client choose the IP to use, this is "random", and
> after can use the same ip for a while... this is not random.

This is a technical mailing list. When you claim that something is
bad, you need to support that claim with actual evidence. Otherwise,
you are just spreading rumors.

How many clients hang on to a server's IP address? How many of
these hang on to the same server IP address? What evidence is the
basis for these claims?

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: load balancing among mail servers

Mark Goodge
In reply to this post by Eero Volotinen-2
On 16/02/2010 15:53, Eero Volotinen wrote:

>
>
> Well, It is bad because server (client) can cache ip address for long
> time and then one of your smtp server fails. It can take long time
> before server gets ip address of working smtp server..

The client is faulty in that case. If it fails to contact the server it
first tries it should then try a different one. And a client which won't
retry when it can't reach a server will have exactly the same problem
even with normal MX records, as it won't fall back to the lower priority
records if the primary is offline.

Mark
--

Please sign my petition: http://petitions.pm.gov.uk/geopostcode
Read why: http://mark.goodge.co.uk/musings/422/locate-that-postcode/
Reply | Threaded
Open this post in threaded view
|

Re: load balancing among mail servers

Patrick Chemla
In reply to this post by jeffrey j donovan
Le 16/02/2010 17:47, donovan jeffrey j a écrit :
> DNS round robin is bad, it works but is defective for real load
> balancing. The client choose the IP to use, this is "random", and
> after can use the same ip for a while... this is not random.
>

Again, I am doing every days exactly what required at the beginning of
this thread.

I have one Postfix server who simply relays all emails to send to a farm
of 40 mail sub-servers.

To load balance, I simply use a local DNS who manage a local domain. All
40 sub-servers are identified as equivalent MX of this local domain. The
Postfix server just ask the DNS what is the MX of this local domain, and
get a name /ip from the DNS.

I am very satisfied of this load balancing.

Again, I said before that the statistics show a difference of less than
2% of the traffic to the sub-server who work the most and the one who
work the less. It is, as I think, very very well load balanced.

When a sub-server fails, some messages are stuck in the Postfix, but a
very small percentage. Immediatly after you restart the sub-server, or
you put it out of the DNS, all messages are processed.

I don't think there is a need for more precision in the load balancing.

I don't think there is a need for keepalive, or any expensive device to
do it.

Patrick

Reply | Threaded
Open this post in threaded view
|

Re: load balancing among mail servers

Jordi Espasa Clofent-5
In reply to this post by angela vulpio
Use  your favorite load-balance app. You have several options in BSD or
GNU/Linux flavors. Personally I use with total success LVS+heartbeat to
load-balance 3 Posfifix (only to send) with 70k account behind.

;)
Reply | Threaded
Open this post in threaded view
|

Re: load balancing among mail servers

Terry Carmen
In reply to this post by Massimo Nuvoli
Quoting Massimo Nuvoli <[hidden email]>:

> donovan jeffrey j ha scritto:
>>
>> On Feb 16, 2010, at 8:09 AM, aa wrote:
>>
>>> Someone advised me to insert in the DNS zone a list of MX records
>>> defined with the same level of priority so the DNS server will choose
>>> one of them without invoking always the same mail server....
>>> It could be an idea, in my opinion, but I'd prefer a "less  random"
>>> solution and a more scientific one....
>> very easy for smtp relays.
>> smtp1
>> smtp2
>> create a dns name smtp, and your system will round robin query for the
>> next available server.
>
> DNS round robin is bad, it works but is defective for real load
> balancing. The client choose the IP to use, this is "random", and
> after can use the same ip for a while... this is not random.
>
> The real solution is lvs or keepalived, the choice of the node is done
> by the load balancer...

A broken client that can't be bothered to check MX records isn't  
anybody's problem except the sender.

In fact, just like spambots, broken clients that send to the wrong MX  
tend to end up on RBLs fairly quickly, making them even less of a  
problem.

Terry

Reply | Threaded
Open this post in threaded view
|

Re: load balancing among mail servers

James R. Marcus-2
In reply to this post by jeffrey j donovan
I did round robin.  Unless your servers are really under a high load I think this is more then suitable.

James


On Feb 16, 2010, at 8:51 AM, donovan jeffrey j wrote:

>
> On Feb 16, 2010, at 8:09 AM, aa wrote:
>
>> Someone advised me to insert in the DNS zone a list of MX records  
>> defined with the same level of priority so the DNS server will  
>> choose one of them without invoking always the same mail server....
>> It could be an idea, in my opinion, but I'd prefer a "less  random"  
>> solution and a more scientific one....
>
> very easy for smtp relays.
>
> smtp1
> smtp2
>
>
> create a dns name smtp, and your system will round robin query for the  
> next available server.

Reply | Threaded
Open this post in threaded view
|

Re: load balancing among mail servers

dealsoc
In reply to this post by Patrick Chemla
Hi everyone in this forum,

In searching for loadbalancing postfix outgoing I find this from Patrick Chemla.

So, Could anyone show me how to implement the Patrick's Model.

get me at dealsocvn@gmail.com

Thanks in advance.
 
Patrick Chemla wrote
Le 16/02/2010 17:47, donovan jeffrey j a écrit :
> DNS round robin is bad, it works but is defective for real load
> balancing. The client choose the IP to use, this is "random", and
> after can use the same ip for a while... this is not random.
>

Again, I am doing every days exactly what required at the beginning of
this thread.

I have one Postfix server who simply relays all emails to send to a farm
of 40 mail sub-servers.

To load balance, I simply use a local DNS who manage a local domain. All
40 sub-servers are identified as equivalent MX of this local domain. The
Postfix server just ask the DNS what is the MX of this local domain, and
get a name /ip from the DNS.

I am very satisfied of this load balancing.

Again, I said before that the statistics show a difference of less than
2% of the traffic to the sub-server who work the most and the one who
work the less. It is, as I think, very very well load balanced.

When a sub-server fails, some messages are stuck in the Postfix, but a
very small percentage. Immediatly after you restart the sub-server, or
you put it out of the DNS, all messages are processed.

I don't think there is a need for more precision in the load balancing.

I don't think there is a need for keepalive, or any expensive device to
do it.

Patrick