Quantcast

Bulk Mailing Performance

classic Classic list List threaded Threaded
33 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Bulk Mailing Performance

Sam Jones
More to satisfy my own curiosity than anything else, I'm wondering about
the performance that could be squeezed out of Postfix in a bulk mailing
capacity.

I have a client that currently uses and ESP who have an astounding
throughput of up to a million messages per hour. This brought up a
discussion about high-performance MTAs and tuning and the general
comments I'm hearing are that things like Postfix, Exim, Sendmail &
are just not man enough for such a task and the absolute best you could
expect from any of them is about 100k messages per hour.

Now, I like to wipe out the fact from fiction because people like
PowerMTA are looking to sell their products and it would be in their
interest to neglect that any MTA (Postfix/Exim/Sendmail) could be set up
in a way that would easily rival their product.

Can anyone on the list tell me if it's possible to performance tune
Postfix to a point where it could complete with this and possible
strategies?

Kind thanks

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bulk Mailing Performance

Robert Schetterer
Am 02.09.2012 11:43, schrieb Sam Jones:

> More to satisfy my own curiosity than anything else, I'm wondering about
> the performance that could be squeezed out of Postfix in a bulk mailing
> capacity.
>
> I have a client that currently uses and ESP who have an astounding
> throughput of up to a million messages per hour. This brought up a
> discussion about high-performance MTAs and tuning and the general
> comments I'm hearing are that things like Postfix, Exim, Sendmail &
> are just not man enough for such a task and the absolute best you could
> expect from any of them is about 100k messages per hour.
>
> Now, I like to wipe out the fact from fiction because people like
> PowerMTA are looking to sell their products and it would be in their
> interest to neglect that any MTA (Postfix/Exim/Sendmail) could be set up
> in a way that would easily rival their product.
>
> Can anyone on the list tell me if it's possible to performance tune
> Postfix to a point where it could complete with this and possible
> strategies?
>
> Kind thanks
>

Hi look on list archives about tuning stuff

also read

http://www.postfix.org/TUNING_README.html

at last , there are two side deliver in/out

for out ,there are things you need to find out about
the recipients  mail servers, cause many of them have limits about get
mail from the same ip adress per time periods,and/or do greylisting and
other filter stuff, so its not only a question how to tune your own
postfix ( which may always be possible to goal your needs, depending on
hardware, filesystems network etc )
for massive bulk mails sometimes multiple instance postfix or a cascade
with smtp fallback postfix servers is used for fasten deliver out mail.
Also possible smtp via loadbalancer etc.

At last there are a lot other recommands ( whitelisting at big mailers ,
use dkim and spf dmarc etc ) doing massive mail which arent related
directly to the mailserver software you use, for sure postfix will never
be the performance problem if setup right to your "massive" needs.
In fact postfix is widly used for massive mail.

--
Best Regards
MfG Robert Schetterer
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bulk Mailing Performance

Viktor Dukhovni
In reply to this post by Sam Jones
On Sun, Sep 02, 2012 at 10:43:07AM +0100, Sam Jones wrote:

> More to satisfy my own curiosity than anything else, I'm wondering about
> the performance that could be squeezed out of Postfix in a bulk mailing
> capacity.

Running a high volume bulk email platform is not a software problem.
It is a logistics problem. Enrolling on the whitelists and feedback
loops of various large email providers, handling bounce-backs,
jumping through rate-limit hoops, ...


> I have a client that currently uses and ESP who have an astounding
> throughput of up to a million messages per hour.

This is not astounding, a single ~2003 Dell 1850 Postfix server
was measured by me at ~300 msgs/sec of deliveries to real users
with nothing but a simple MegaRAID controller (with battery cache)
striping two SCSI disks. This would go another factor of 2 faster
on today's commodity servers, but the real issue is finding peers
who'll accept your mail at that rate.

> discussion about high-performance MTAs and tuning and the general
> comments I'm hearing are that things like Postfix, Exim, Sendmail &
> are just not man enough for such a task and the absolute best you could
> expect from any of them is about 100k messages per hour.

Many bulk email services in fact use Postfix, Exim, ...  The MTA
software is often not the bottleneck. They split bulk deliveries
over many machines (or lots of IPs on the same machine) and tune
to avoid throttling by the ESP over and above all other concerns.
Raw MTA performance is rarely a factor.

> Now, I like to wipe out the fact from fiction because people like
> PowerMTA are looking to sell their products and it would be in their
> interest to neglect that any MTA (Postfix/Exim/Sendmail) could be set up
> in a way that would easily rival their product.

Bulk email is a logistics exercise. When you choose an bulk email
delivery service, you're buying their logistics skills and their
reputation with mailbox providers (Gmail, Hotmail, Yahoo, ...)

> Can anyone on the list tell me if it's possible to performance tune
> Postfix to a point where it could complete with this and possible
> strategies?

Wrong question.

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bulk Mailing Performance

Sam Jones
On Sun, 2012-09-02 at 15:39 +0000, Viktor Dukhovni wrote:

> On Sun, Sep 02, 2012 at 10:43:07AM +0100, Sam Jones wrote:
>
> > More to satisfy my own curiosity than anything else, I'm wondering about
> > the performance that could be squeezed out of Postfix in a bulk mailing
> > capacity.
>
> Running a high volume bulk email platform is not a software problem.
> It is a logistics problem. Enrolling on the whitelists and feedback
> loops of various large email providers, handling bounce-backs,
> jumping through rate-limit hoops, ...
>
>
> > I have a client that currently uses and ESP who have an astounding
> > throughput of up to a million messages per hour.
>
> This is not astounding, a single ~2003 Dell 1850 Postfix server
> was measured by me at ~300 msgs/sec of deliveries to real users
> with nothing but a simple MegaRAID controller (with battery cache)
> striping two SCSI disks. This would go another factor of 2 faster
> on today's commodity servers, but the real issue is finding peers
> who'll accept your mail at that rate.
>
> > discussion about high-performance MTAs and tuning and the general
> > comments I'm hearing are that things like Postfix, Exim, Sendmail &
> > are just not man enough for such a task and the absolute best you could
> > expect from any of them is about 100k messages per hour.
>
> Many bulk email services in fact use Postfix, Exim, ...  The MTA
> software is often not the bottleneck. They split bulk deliveries
> over many machines (or lots of IPs on the same machine) and tune
> to avoid throttling by the ESP over and above all other concerns.
> Raw MTA performance is rarely a factor.
>
> > Now, I like to wipe out the fact from fiction because people like
> > PowerMTA are looking to sell their products and it would be in their
> > interest to neglect that any MTA (Postfix/Exim/Sendmail) could be set up
> > in a way that would easily rival their product.
>
> Bulk email is a logistics exercise. When you choose an bulk email
> delivery service, you're buying their logistics skills and their
> reputation with mailbox providers (Gmail, Hotmail, Yahoo, ...)
>
> > Can anyone on the list tell me if it's possible to performance tune
> > Postfix to a point where it could complete with this and possible
> > strategies?
>
> Wrong question.
>
I really appreciate the post Viktor. Thought provoking and clear.

I guess what I'm querying in a way is some of the sales blurb from
people like PowerMTA & GreenArrow and the remarks they make about open
source solutions like Postfix etc. This one in particular: "Open source
Mail Transfer Agents (MTAs) often max out between 20 and 30 thousand
messages per hour. GreenArrow can send 300,000 messages per hour—more
than ten times as fast."

If we strip this back to hypothetical and assume a perfect world without
any issues like rate limiting and rejection, small emails with nomay
restrictions or network issues with the recipient MX's, is the above
statement plausibly true?

I'm assuming - and I've not yet looked deeply at this - that there is
probably a way to get Postfix to run parallel instances to improve
delivery speed.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bulk Mailing Performance

Wietse Venema
Sam Jones:
> I guess what I'm querying in a way is some of the sales blurb from
> people like PowerMTA & GreenArrow and the remarks they make about open
> source solutions like Postfix etc. This one in particular: "Open source
> Mail Transfer Agents (MTAs) often max out between 20 and 30 thousand
> messages per hour.

That's 7 deliveries per second, a number that you might get with
an MTA that makes one connection at a time (Sendmail, Exim).

You get larger numbers with parallel deliveries. Those are built-into
Postfix/qmail/etc.  With MTAs such as Sendmail/Exim/etc. you get
parallel deliveries by using parallel submission.

Once you reach the file system performance limit, Postfix/qmail/etc.
require parallel submission as well, with different MTAs running
on top of independent file systems.

As Victor says, it is not primarily an MTA problem. The technical
problems are a matter of proper diagnosis and appropriate tuning.

        Wietse
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bulk Mailing Performance

Lorens Kockum-2
In reply to this post by Sam Jones
The exact same question was sent by someone calling himself
"Ron White" to the exim mailing list at almost exactly the same
time. Peddling one's services by soliciting comparisons with
competitors is so passé . . .

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bulk Mailing Performance

John Peach-2
On Sun, 2 Sep 2012 22:46:10 +0200
Lorens Kockum <[hidden email]> wrote:

> The exact same question was sent by someone calling himself
> "Ron White" to the exim mailing list at almost exactly the same
> time. Peddling one's services by soliciting comparisons with
> competitors is so passé . . .

I find it rather useful; lets me know what I should be blocking....
>


--
John
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bulk Mailing Performance

Dario "subbia" Cavallaro
In reply to this post by Sam Jones
Il 02/09/2012 11:43, Sam Jones ha scritto:

> More to satisfy my own curiosity than anything else, I'm wondering about
> the performance that could be squeezed out of Postfix in a bulk mailing
> capacity.
>
> I have a client that currently uses and ESP who have an astounding
> throughput of up to a million messages per hour. This brought up a
> discussion about high-performance MTAs and tuning and the general
> comments I'm hearing are that things like Postfix, Exim, Sendmail &
> are just not man enough for such a task and the absolute best you could
> expect from any of them is about 100k messages per hour.
>
> Now, I like to wipe out the fact from fiction because people like
> PowerMTA are looking to sell their products and it would be in their
> interest to neglect that any MTA (Postfix/Exim/Sendmail) could be set up
> in a way that would easily rival their product.
>
> Can anyone on the list tell me if it's possible to performance tune
> Postfix to a point where it could complete with this and possible
> strategies?
>
> Kind thanks
>
>
In my experience, I have never had problems with performances in the
past with Postfix. Though I had stability issues, but just because we
reached out the limit of opened files (OS tuning). And it was during a
spam attack (probably around a milion email per hour during attack).

Hope this helps.

Cheers,
Dario "subbia" Cavallaro
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bulk Mailing Performance

Viktor Dukhovni
In reply to this post by Sam Jones
On Sun, Sep 02, 2012 at 07:14:35PM +0100, Sam Jones wrote:

> I guess what I'm querying in a way is some of the sales blurb from
> people like PowerMTA & GreenArrow and the remarks they make about open
> source solutions like Postfix etc. This one in particular: "Open source
> Mail Transfer Agents (MTAs) often max out between 20 and 30 thousand
> messages per hour. GreenArrow can send 300,000 messages per hour?more
> than ten times as fast."

As I said, I measured 300 msgs/sec with Postfix on relatively modest
hardware in 2003. This is not too difficult, just configure sufficient
output concurrency, and provide a low latency disk (battery cache
striped RAID).

With RAM disk (a queue-manager bottleneck analysis, circa five
years ago) Postfix yielded ~3000 msgs/sec on a dual Opteron box
delivering to the discard transport. So that's your ceiling if you
provide sufficient disk and network bandwidth, eventually the queue
manager runs out of CPU, but this is at rates approaching 10 million
messages an hour.

The throughput numbers are not that interesting anymore, I go for
reliability, security and flexibility. I also go for a solid
architecture that degrades well under load, and that's why I
really like Postfix, but this is a difficult point to make,
most people are not in a position to understand why Postfix
stands out in this regard.

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bulk Mailing Performance

DTNX Postmaster
On Sep 3, 2012, at 03:56, Viktor Dukhovni wrote:

> On Sun, Sep 02, 2012 at 07:14:35PM +0100, Sam Jones wrote:
>
>> I guess what I'm querying in a way is some of the sales blurb from
>> people like PowerMTA & GreenArrow and the remarks they make about open
>> source solutions like Postfix etc. This one in particular: "Open source
>> Mail Transfer Agents (MTAs) often max out between 20 and 30 thousand
>> messages per hour. GreenArrow can send 300,000 messages per hour?more
>> than ten times as fast."
>
> As I said, I measured 300 msgs/sec with Postfix on relatively modest
> hardware in 2003. This is not too difficult, just configure sufficient
> output concurrency, and provide a low latency disk (battery cache
> striped RAID).
>
> With RAM disk (a queue-manager bottleneck analysis, circa five
> years ago) Postfix yielded ~3000 msgs/sec on a dual Opteron box
> delivering to the discard transport. So that's your ceiling if you
> provide sufficient disk and network bandwidth, eventually the queue
> manager runs out of CPU, but this is at rates approaching 10 million
> messages an hour.
>
> The throughput numbers are not that interesting anymore, I go for
> reliability, security and flexibility. I also go for a solid
> architecture that degrades well under load, and that's why I
> really like Postfix, but this is a difficult point to make,
> most people are not in a position to understand why Postfix
> stands out in this regard.

In other words, if 'we strip this back to hypothetical and assume a
perfect world without any issues', this 'GreenArrow' maxes out at
300,000 messages per hour. Postfix can send 10,8 million messages per
hour, more than 35 times as fast*.

Lies, damn lies, and vendor benchmarks, heh.

Cya,
Jona

--

* Tests performed in an optimized lab environment. Operational
  restrictions may apply in real world environments.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bulk Mailing Performance

Stefan Foerster-2
In reply to this post by Sam Jones
* Sam Jones <[hidden email]>:
> I guess what I'm querying in a way is some of the sales blurb from
> people like PowerMTA & GreenArrow and the remarks they make about open
> source solutions like Postfix etc. This one in particular: "Open source
> Mail Transfer Agents (MTAs) often max out between 20 and 30 thousand
> messages per hour. GreenArrow can send 300,000 messages per hour—more
> than ten times as fast."

My desktop machine - the very same one I'm writing this mail on - is
currently delivering 65 messages per second to a smtp-sink. On a
single 2TB SATA drive that will probably suffer a heart attack if you
even so much as threaten it with more than 20 IOPS/s. That's while
watching a 1080p video (footage from my last holiday). With no tuning
whatsoever.

Granted, not a real world use case, but still, don't trust everything
you read on the web.


Stefan
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bulk Mailing Performance

Ralf Hildebrandt
In reply to this post by Sam Jones
* Sam Jones <[hidden email]>:

> More to satisfy my own curiosity than anything else, I'm wondering about
> the performance that could be squeezed out of Postfix in a bulk mailing
> capacity.

The problem is mostly on the receiving side, when the receiving system
starts throtteling you.

> I have a client that currently uses and ESP who have an astounding
> throughput of up to a million messages per hour. This brought up a
> discussion about high-performance MTAs and tuning and the general
> comments I'm hearing are that things like Postfix, Exim, Sendmail &
> are just not man enough for such a task and the absolute best you could
> expect from any of them is about 100k messages per hour.

I once sent 2096/min*60min = 125.760mails/minute on mail.python.org
and there the generation of the mails is the limiting factor.

--
Ralf Hildebrandt
  Geschäftsbereich IT | Abteilung Netzwerk
  Charité - Universitätsmedizin Berlin
  Campus Benjamin Franklin
  Hindenburgdamm 30 | D-12203 Berlin
  Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
  [hidden email] | http://www.charite.de
           
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bulk Mailing Performance

Ralf Hildebrandt
In reply to this post by Viktor Dukhovni
* Viktor Dukhovni <[hidden email]>:

> Running a high volume bulk email platform is not a software problem.
> It is a logistics problem. Enrolling on the whitelists and feedback
> loops of various large email providers, handling bounce-backs,
> jumping through rate-limit hoops, ...

Absolutely.

--
Ralf Hildebrandt
  Geschäftsbereich IT | Abteilung Netzwerk
  Charité - Universitätsmedizin Berlin
  Campus Benjamin Franklin
  Hindenburgdamm 30 | D-12203 Berlin
  Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
  [hidden email] | http://www.charite.de
           
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bulk Mailing Performance

Stan Hoeppner
In reply to this post by DTNX Postmaster
On 9/3/2012 12:02 AM, DTNX Postmaster wrote:

> In other words, if 'we strip this back to hypothetical and assume a
> perfect world without any issues', this 'GreenArrow' maxes out at
> 300,000 messages per hour. Postfix can send 10,8 million messages per
> hour, more than 35 times as fast*.

In all fairness, given your "perfect world" criteria, this ESP would be
moving a lot more mail as well, with no restrictions on the outbound
pipe or at the receiver.

But as others have correctly pointed out, the issue here isn't MTA
performance, it's administrative performance.  The last thread I
responded to demonstrates this.  The big advantage ESPs have is their
established relationships with the freemailers and other large mailbox
providers.  These allow them greater throughput than the unwashed bulk
sender, at least into the receiver's initial queue.

--
Stan

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bulk Mailing Performance

DTNX Postmaster
On Sep 3, 2012, at 13:05, Stan Hoeppner wrote:

> On 9/3/2012 12:02 AM, DTNX Postmaster wrote:
>
>> In other words, if 'we strip this back to hypothetical and assume a
>> perfect world without any issues', this 'GreenArrow' maxes out at
>> 300,000 messages per hour. Postfix can send 10,8 million messages per
>> hour, more than 35 times as fast*.
>
> In all fairness, given your "perfect world" criteria, this ESP would be
> moving a lot more mail as well, with no restrictions on the outbound
> pipe or at the receiver.
>
> But as others have correctly pointed out, the issue here isn't MTA
> performance, it's administrative performance.  The last thread I
> responded to demonstrates this.  The big advantage ESPs have is their
> established relationships with the freemailers and other large mailbox
> providers.  These allow them greater throughput than the unwashed bulk
> sender, at least into the receiver's initial queue.

They aren't my perfect world criteria, but a direct quote from Sam
Jones' earlier buzzword compliant reply.

It was meant to illustrate the often ridiculous nature of vendor
benchmarks, how useless they are in real world situations, and
therefore how silly it is to pick software based on theoretical limits
you will most likely never hit.

Not enough sarcasm, I guess ;-)

Cya,
Jona

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bulk Mailing Performance

Jose-Marcio Martins da Cruz
DTNX Postmaster wrote:

>
> They aren't my perfect world criteria, but a direct quote from Sam
> Jones' earlier buzzword compliant reply.
>
> It was meant to illustrate the often ridiculous nature of vendor
> benchmarks, how useless they are in real world situations, and
> therefore how silly it is to pick software based on theoretical limits
> you will most likely never hit.

Not really ridiculous. All those benchmarks are interesting, as they represent,
say, the "intrinsic performance of the software". The problem is to tell (for
the vendor) and to take into account (for the reader) the conditions at which
the benchmark was done.

But, sure, two pieces of software can be compared only if measurings are done
with the same conditions. And one software which has better "intrinsic
performance" may not be better in real world conditions.


--

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bulk Mailing Performance

Sam Jones
In reply to this post by Lorens Kockum-2
On Sun, 2012-09-02 at 22:46 +0200, Lorens Kockum wrote:
> The exact same question was sent by someone calling himself
> "Ron White" to the exim mailing list at almost exactly the same
> time. Peddling one's services by soliciting comparisons with
> competitors is so passé . . .
>
Yes, it was. Well done. The question applied to both MTA's and funny
enough, the use of Aliases on the internet is nothing new.

Thanks to those that contributed useful information. I think it's safe
to say that the sales blurb is looking at a very basic scenario.



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bulk Mailing Performance

Daniel L. Miller
In reply to this post by Sam Jones
On 9/2/2012 11:14 AM, Sam Jones wrote:

> On Sun, 2012-09-02 at 15:39 +0000, Viktor Dukhovni wrote:
>> On Sun, Sep 02, 2012 at 10:43:07AM +0100, Sam Jones wrote:
>>
>>> More to satisfy my own curiosity than anything else, I'm wondering about
>>> the performance that could be squeezed out of Postfix in a bulk mailing
>>> capacity.
>> Running a high volume bulk email platform is not a software problem.
>> It is a logistics problem. Enrolling on the whitelists and feedback
>> loops of various large email providers, handling bounce-backs,
>> jumping through rate-limit hoops, ...
[...]
>>
>> I guess what I'm querying in a way is some of the sales blurb from
>> people like PowerMTA & GreenArrow and the remarks they make about open
>> source solutions like Postfix etc. This one in particular: "Open source
>> Mail Transfer Agents (MTAs) often max out between 20 and 30 thousand
>> messages per hour. GreenArrow can send 300,000 messages per hour—more
>> than ten times as fast."
>>

Knowing absolutely nothing about the software mentioned - I would say
there is a difference between messages SENT vs messages DELIVERED.  I
realize many will immediately correct me and say even Postfix can't
guarantee delivery to a given recipient - merely acknowledgement of the
recipient server's acceptance - but I don't know how else to
discriminate between a single-pass of a message, without retries,
without verification, without greylist tolerance, without reporting,
just knock on the door and try to shove it on - vs reliable message
handling.

Again, knowing nothing about alternatives to Postfix - I question
whether software intended for bulk mailing purposes is designed in such
a manner.  As a crude analogy, even the best machine gun doesn't have a
fraction of the accuracy of a quality sniper rifle - but on the other
hand a machine gun will put a lot more lead downrange.  Different tools
for different purposes.  Spray-and-Pray - or deliver the personal message.

--
Daniel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bulk Mailing Performance

fletch
In reply to this post by Sam Jones
The postfix performance claims made via this thread are far-fetched to say the least.  Most postfix users will only see outbound throughput in the range of ~250,000/hour per instance in a production setting.  Yet, people on here are claiming 10 million/hour?  I guess that would be possible if a sender were to run, say, 40 postfix instances which would be a complete management nightmare of course.

Obviously, vendors like Port25 (company behind PowerMTA) and GreenArrow would not be able to make any sales if the benefits of commercial software products v. open source were not substantial.  
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bulk Mailing Performance

Wietse Venema
fletch:

> The postfix performance claims made via this thread are far-fetched to say
> the least.  Most postfix users will only see outbound throughput in the
> range of ~250,000/hour per instance in a production setting.  Yet, people on
> here are claiming 10 million/hour?  I guess that would be possible if a
> sender were to run, say, 40 postfix instances which would be a complete
> management nightmare of course.
>
> Obviously, vendors like Port25 (company behind PowerMTA) and GreenArrow
> would not be able to make any sales if the benefits of commercial software
> products v. open source were not substantial.  

Is this a troll?

I have documented Postfix performance claims on Wikipedia. Be sure
to read the cautionary note about factors outside of Postfix that
in practice limit the delivery performance.

        Wietse
12
Loading...