Postfix/milter benchmarking

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Postfix/milter benchmarking

Julian Mehnle
Dear Postfix community,

I'm developing a new milter and I need to benchmark it when plugged into Postfix. What are my best options for generating an SMTP stream of messages directed at a single Postfix instance at rates on the order of hundreds per second, and measuring SMTP acceptance rates and SMTP response latency? I can run processes on multiple machines if necessary, but the more efficient the better.

I saw the smtp-source tool <http://www.postfix.org/smtp-source.1.html>, but I'm not sure I can tune it to certain output rates and make it take acceptance rate and latency measurements.

Are there any other tools people use to benchmark their Postfix setups or, more specifically, milters?

Thanks,

-Julian

Reply | Threaded
Open this post in threaded view
|

Re: Postfix/milter benchmarking

Wietse Venema
Julian Mehnle:

> Dear Postfix community,
>
> I'm developing a new milter and I need to benchmark it when plugged
> into Postfix. What are my best options for generating an SMTP
> stream of messages directed at a single Postfix instance at rates
> on the order of hundreds per second, and measuring SMTP acceptance
> rates and SMTP response latency? I can run processes on multiple
> machines if necessary, but the more efficient the better.
>
> I saw the smtp-source tool <http://www.postfix.org/smtp-source.1.html>,
> but I'm not sure I can tune it to certain output rates and make
> it take acceptance rate and latency measurements.

Try it and see how fast it goes with and without your Milter.

> Are there any other tools people use to benchmark their Postfix
> setups or, more specifically, milters?

Have you tried "smtp benchmark" as a search engine query? I have
not used any of the programs out there.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Postfix/milter benchmarking

Patrick Ben Koetter-2
In reply to this post by Julian Mehnle
* Julian Mehnle <[hidden email]>:
> I'm developing a new milter and I need to benchmark it when plugged into
> Postfix. What are my best options for generating an SMTP stream of messages
> directed at a single Postfix instance at rates on the order of hundreds per
> second, and measuring SMTP acceptance rates and SMTP response latency? I can
> run processes on multiple machines if necessary, but the more efficient the
> better.

mstone is a PITA to setup, but it works.
swaks is a script, which you can create a wrapper around.

Which fits better? Depends on the job your milter needs to do. What is its
focus? If try to filter sessions, mstone will probably serve you better, since
you can easily tune frequency, number of connections etc. If you need to
simulate specific content swaks might be better.

We've easily done 90 msg/sec with a milter that picks up session data and
specific headers on standard server hardware.

p@rick



>
> I saw the smtp-source tool <http://www.postfix.org/smtp-source.1.html>, but I'm not sure I can tune it to certain output rates and make it take acceptance rate and latency measurements.
>
> Are there any other tools people use to benchmark their Postfix setups or, more specifically, milters?
>
> Thanks,
>
> -Julian
>

--
[*] sys4 AG
 
https://sys4.de, +49 (89) 30 90 46 64
Franziskanerstraße 15, 81669 München
 
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
 
Reply | Threaded
Open this post in threaded view
|

Re: Postfix/milter benchmarking

A. Schulze
In reply to this post by Julian Mehnle

Julian Mehnle:

> Are there any other tools people use to benchmark their Postfix  
> setups or, more specifically, milters?

Wietse told on talk I listened
"Optimize both the worst case and the common case. Worst cases become  
normal cases"

When developing milter applicaions that mean to me:
  - decide as early as possible
  - allocate memory / do complex calculation as late as possible

Andreas

Reply | Threaded
Open this post in threaded view
|

Re: Postfix/milter benchmarking

Kouhei Sutou-2
In reply to this post by Julian Mehnle
Hi,

In <[hidden email]>
  "Postfix/milter benchmarking" on Thu, 23 Oct 2014 08:45:06 -0700,
  Julian Mehnle <[hidden email]> wrote:

> I'm developing a new milter and I need to benchmark it when plugged into Postfix. What are my best options for generating an SMTP stream of messages directed at a single Postfix instance at rates on the order of hundreds per second, and measuring SMTP acceptance rates and SMTP response latency? I can run processes on multiple machines if necessary, but the more efficient the better.
...
> Are there any other tools people use to benchmark their Postfix setups or, more specifically, milters?

I'm developing some milters. I created two tools to
benchmark and test my milters:

  * milter-test-server:
    http://milter-manager.sourceforge.net/reference/milter-test-server.html

    It talks milter protocol. It means that you don't need
    Postfix, you just need milter.

    It just does one milter session and reports elapsed
    time.

  * milter-performance-check:
    http://milter-manager.sourceforge.net/reference/milter-performance-check.html

    It talks SMTP. You can measure elapsed time for
    Postfix + milter.

    It sends multiple mails and reports performance
    statistics. See the above URL for details.

See the following URL how to install these tools:
  http://milter-manager.sourceforge.net/reference/install-to.html


Thanks,
--
kou