postfix milter body chunk length

classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|

postfix milter body chunk length

Matthias Schneider
Hi,

I was wondering why the transfer of a 100mb mail to my milter
application was slow, i found the bottleneck in the body chunk transfer.

The maximum packet length seems to be fixed to 64k, it would be great if
we could make that configurable in postfix (uint32 is possible).


best regards,

Matthias Schneider

Reply | Threaded
Open this post in threaded view
|

Re: postfix milter body chunk length

Viktor Dukhovni
> On Aug 16, 2019, at 10:33 PM, Matthias Schneider <[hidden email]> wrote:
>
> I was wondering why the transfer of a 100mb mail to my milter application was slow, i found the bottleneck in the body chunk transfer.
>
> The maximum packet length seems to be fixed to 64k, it would be great if we could make that configurable in postfix (uint32 is possible).

https://github.com/avar/sendmail-pmilter/blob/master/doc/milter-protocol.txt#L182-L213

http://ftp.sendmail.org/KNOWNBUGS

* milter communication fails if a single header is larger than 64K.

  If a single header is larger than 64KB (which is not possible in the
  default configuration) then it cannot be transferred in one block to
  libmilter and hence the communication fails.  This can be avoided by
  increasing the constant MILTER_CHUNK_SIZE in
  include/libmilter/mfdef.h and recompiling sendmail, libmilter, and
  all (statically linked) milters (or by using undocumented compile
  time options: _FFR_MAXDATASIZE/_FFR_MDS_NEGOTIATE; you have to
  read the source code in order to use these properly).

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: postfix milter body chunk length

Wietse Venema
Viktor Dukhovni:

> > On Aug 16, 2019, at 10:33 PM, Matthias Schneider <[hidden email]> wrote:
> >
> > I was wondering why the transfer of a 100mb mail to my milter application was slow, i found the bottleneck in the body chunk transfer.
> >
> > The maximum packet length seems to be fixed to 64k, it would be great if we could make that configurable in postfix (uint32 is possible).
>
> https://github.com/avar/sendmail-pmilter/blob/master/doc/milter-protocol.txt#L182-L213
>
> http://ftp.sendmail.org/KNOWNBUGS
>
> * milter communication fails if a single header is larger than 64K.
>
>   If a single header is larger than 64KB (which is not possible in the
>   default configuration) then it cannot be transferred in one block to
>   libmilter and hence the communication fails.  This can be avoided by
>   increasing the constant MILTER_CHUNK_SIZE in
>   include/libmilter/mfdef.h and recompiling sendmail, libmilter, and
>   all (statically linked) milters (or by using undocumented compile
>   time options: _FFR_MAXDATASIZE/_FFR_MDS_NEGOTIATE; you have to
>   read the source code in order to use these properly).

On the Postfix side, edit src/milter/milter8.c and update its
MILTER_CHUNK_SIZE definition accordingly. It's only compile-time
configurable in Postfix, because it's only compile-time configurable
in libmilter.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: postfix milter body chunk length

Matthias Schneider
In reply to this post by Viktor Dukhovni
Any chance to get a config for MILTER_CHUNK_SIZE? It would be great so
have an easy and fast switch. I may have different settings on different
mail systems (because of 3rd party milters), its hard to manage
different customized postfix versions in my distribution repositories.


Thanks,

Matthias Schneider

Am 16.08.19 um 15:32 schrieb Matthias Schneider:

> Any chance to get a config for MILTER_CHUNK_SIZE? It would be great so
> have an easy and fast switch. I may have different settings on
> different mail systems (because of 3rd party milters), its hard to
> manage different customized postfix versions in my distribution
> repositories.
>
>
> Thanks,
>
> Matthias Schneider
>
>
>
> Am 16.08.19 um 14:49 schrieb Viktor Dukhovni:
>>> On Aug 16, 2019, at 10:33 PM, Matthias Schneider
>>> <[hidden email]> wrote:
>>>
>>> I was wondering why the transfer of a 100mb mail to my milter
>>> application was slow, i found the bottleneck in the body chunk
>>> transfer.
>>>
>>> The maximum packet length seems to be fixed to 64k, it would be
>>> great if we could make that configurable in postfix (uint32 is
>>> possible).
>> * milter communication fails if a single header is larger than 64K.
>>
>>    If a single header is larger than 64KB (which is not possible in the
>>    default configuration) then it cannot be transferred in one block to
>>    libmilter and hence the communication fails.  This can be avoided by
>>    increasing the constant MILTER_CHUNK_SIZE in
>>    include/libmilter/mfdef.h and recompiling sendmail, libmilter, and
>>    all (statically linked) milters (or by using undocumented compile
>>    time options: _FFR_MAXDATASIZE/_FFR_MDS_NEGOTIATE; you have to
>>    read the source code in order to use these properly).
>>
Reply | Threaded
Open this post in threaded view
|

Re: postfix milter body chunk length

Viktor Dukhovni
On Fri, Aug 16, 2019 at 03:51:05PM +0200, Matthias Schneider wrote:

> Any chance to get a config for MILTER_CHUNK_SIZE? It would be great so
> have an easy and fast switch. I may have different settings on different
> mail systems (because of 3rd party milters), its hard to manage
> different customized postfix versions in my distribution repositories.

You could raise the limit, but it may be more productive to figure
out why your milter application is "slow" with the default chunk
size.  AFAIK, other milter applications are doing fine with the
default size.

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: postfix milter body chunk length

Matthias Schneider
In reply to this post by Wietse Venema
my tests are little bit frustrating:


Postfix with default milter body chunk size 65535:

mail processing time 1m30.154298259s

Postfix with milter body chunk size 1048576:

mail processing time 17.52360866s


(same mail of 107421kb size / some time is added in body() but it should
be comparable)


Best regards

Matthias Schneider



Am 16.08.19 um 15:50 schrieb Wietse Venema:
>
> On the Postfix side, edit src/milter/milter8.c and update its
> MILTER_CHUNK_SIZE definition accordingly. It's only compile-time
> configurable in Postfix, because it's only compile-time configurable
> in libmilter.
>
> Wietse
Reply | Threaded
Open this post in threaded view
|

Re: postfix milter body chunk length

Matthias Schneider
In reply to this post by Viktor Dukhovni
Am 16.08.19 um 16:03 schrieb Viktor Dukhovni:
> You could raise the limit, but it may be more productive to figure
> out why your milter application is "slow" with the default chunk
> size.  AFAIK, other milter applications are doing fine with the
> default size.
>
I am using the golang implementation:

https://github.com/mschneider82/milter/blob/master/session.go#L288

I think the problem is that every chunk needs to be accepted by the milter.

109.999.104 bytes / 65535 = 1679 accepts


I have the same speed issue in the java library too.

Best regards

Matthias Schneider



Reply | Threaded
Open this post in threaded view
|

Re: postfix milter body chunk length

Wietse Venema
In reply to this post by Matthias Schneider
Matthias Schneider:
> my tests are little bit frustrating:
>
>
> Postfix with default milter body chunk size 65535:
>
> mail processing time 1m30.154298259s
>
> Postfix with milter body chunk size 1048576:

16x larger packet size.

> mail processing time 17.52360866s

5x faster processing. You can get 4x with a much smaller
packet size. But who gives a * about memory these days.

        Wietse

> (same mail of 107421kb size / some time is added in body() but it should
> be comparable)
>
>
> Best regards
>
> Matthias Schneider
>
>
>
> Am 16.08.19 um 15:50 schrieb Wietse Venema:
> >
> > On the Postfix side, edit src/milter/milter8.c and update its
> > MILTER_CHUNK_SIZE definition accordingly. It's only compile-time
> > configurable in Postfix, because it's only compile-time configurable
> > in libmilter.
> >
> > Wietse
>
Reply | Threaded
Open this post in threaded view
|

Re: postfix milter body chunk length

Wietse Venema
In reply to this post by Matthias Schneider
Matthias Schneider:
> Any chance to get a config for MILTER_CHUNK_SIZE? It would be great so
> have an easy and fast switch. I may have different settings on different
> mail systems (because of 3rd party milters), its hard to manage
> different customized postfix versions in my distribution repositories.

Why? It's a compile-time constant in sendmail/libmilter. In Postfix,
MILTER_CHUNK_SIZE is not the only constant to limit packet sizes.
There is also the XXX_MAX_DATA sanity limit for responses from the
Milters, that will trigger a panic if the size is exceeded (that
should probably be just a protocol error). All that stuff would
need to be updated to make the packet size configurable without
introducing unstable edge cases.

        Wietse
 

>
> Thanks,
>
> Matthias Schneider
>
> Am 16.08.19 um 15:32 schrieb Matthias Schneider:
> > Any chance to get a config for MILTER_CHUNK_SIZE? It would be great so
> > have an easy and fast switch. I may have different settings on
> > different mail systems (because of 3rd party milters), its hard to
> > manage different customized postfix versions in my distribution
> > repositories.
> >
> >
> > Thanks,
> >
> > Matthias Schneider
> >
> >
> >
> > Am 16.08.19 um 14:49 schrieb Viktor Dukhovni:
> >>> On Aug 16, 2019, at 10:33 PM, Matthias Schneider
> >>> <[hidden email]> wrote:
> >>>
> >>> I was wondering why the transfer of a 100mb mail to my milter
> >>> application was slow, i found the bottleneck in the body chunk
> >>> transfer.
> >>>
> >>> The maximum packet length seems to be fixed to 64k, it would be
> >>> great if we could make that configurable in postfix (uint32 is
> >>> possible).
> >> * milter communication fails if a single header is larger than 64K.
> >>
> >> ?? If a single header is larger than 64KB (which is not possible in the
> >> ?? default configuration) then it cannot be transferred in one block to
> >> ?? libmilter and hence the communication fails.? This can be avoided by
> >> ?? increasing the constant MILTER_CHUNK_SIZE in
> >> ?? include/libmilter/mfdef.h and recompiling sendmail, libmilter, and
> >> ?? all (statically linked) milters (or by using undocumented compile
> >> ?? time options: _FFR_MAXDATASIZE/_FFR_MDS_NEGOTIATE; you have to
> >> ?? read the source code in order to use these properly).
> >>
>

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: postfix milter body chunk length

Chris Wedgwood
In reply to this post by Matthias Schneider
> Postfix with default milter body chunk size 65535:
>
> mail processing time 1m30.154298259s
>
> Postfix with milter body chunk size 1048576:
>
> mail processing time 17.52360866s

it looks to me like postfix is able to feed a milter very quickly

i just did a couple of quick tests here, an ~83 MiB message only takes
a second or two to pass through two milters (debug log shows 1360 64K
fragments and one smaller tail fragment)
Reply | Threaded
Open this post in threaded view
|

Re: postfix milter body chunk length

Matthias Schneider
Chris, can you tell me your postfix version/settings?

When i do a load-test on my milter application, i can also get 100mb in
about a second proceesed (with 65535 sized chunks).

But postfix (3.3.0 and 3.4.5) only sends about 24 body chunks per second
to my milter application. Its the only milter configured and the milter
is running on 127.0.0.1 (so no latency issues)

Matthias




Am 16.08.19 um 18:06 schrieb Chris Wedgwood:

>> Postfix with default milter body chunk size 65535:
>>
>> mail processing time 1m30.154298259s
>>
>> Postfix with milter body chunk size 1048576:
>>
>> mail processing time 17.52360866s
> it looks to me like postfix is able to feed a milter very quickly
>
> i just did a couple of quick tests here, an ~83 MiB message only takes
> a second or two to pass through two milters (debug log shows 1360 64K
> fragments and one smaller tail fragment)
Reply | Threaded
Open this post in threaded view
|

Re: postfix milter body chunk length

Matthias Schneider
After switching from tcp to unix socket, postfix as milter client is
fast like a rocket (less than a second).

Just wondering which tcp parameter causes the long delay


Matthias Schneider



Am 19.08.19 um 10:34 schrieb Matthias Schneider:

> Chris, can you tell me your postfix version/settings?
>
> When i do a load-test on my milter application, i can also get 100mb
> in about a second proceesed (with 65535 sized chunks).
>
> But postfix (3.3.0 and 3.4.5) only sends about 24 body chunks per
> second to my milter application. Its the only milter configured and
> the milter is running on
> https://protection.retarus.com/v1?u=127.0.0.1&c=3ii9Gwp&r=3FiFpNbrhgwKRK1FMoFWR1&k=7s1&s=2xmP97Jog50dON88kIi5CzD5MGfUsDQvVF2HIwJfHq4 
> (so no latency issues)
>
> Matthias
>
>
>
>
> Am 16.08.19 um 18:06 schrieb Chris Wedgwood:
>>> Postfix with default milter body chunk size 65535:
>>>
>>> mail processing time 1m30.154298259s
>>>
>>> Postfix with milter body chunk size 1048576:
>>>
>>> mail processing time 17.52360866s
>> it looks to me like postfix is able to feed a milter very quickly
>>
>> i just did a couple of quick tests here, an ~83 MiB message only takes
>> a second or two to pass through two milters (debug log shows 1360 64K
>> fragments and one smaller tail fragment)
Reply | Threaded
Open this post in threaded view
|

Re: postfix milter body chunk length

Wietse Venema
In reply to this post by Matthias Schneider
Matthias Schneider:
> Chris, can you tell me your postfix version/settings?
>
> When i do a load-test on my milter application, i can also get 100mb in
> about a second proceesed (with 65535 sized chunks).
>
> But postfix (3.3.0 and 3.4.5) only sends about 24 body chunks per second
> to my milter application. Its the only milter configured and the milter
> is running on 127.0.0.1 (so no latency issues)

Obviously, Postfix can send faster than that, as witnessed by various
people on the mailing list. The Postfix Milter implementation has
not changed in years.

I suspect that the bottleneck is on the receiving side.

- Maybe the loopback stack does not like the 65535 block size. Try
using an ethernet interface address instead.

- Maybe the application "appends" the new content to the end of a
large string; that requires allocating and copying huge amounts of
memory each time. That alone may explain the difference in program
speed.

- Maybe the above program behavior triggers a worst case in the
malloc library as it needs to deal with copies of strings of
increasing length.

You can test that with a test program that does nothing with the
data, such as the test-milter program that is part of Postfix
source code.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: postfix milter body chunk length

Wietse Venema
In reply to this post by Matthias Schneider
Matthias Schneider:
> After switching from tcp to unix socket, postfix as milter client is
> fast like a rocket (less than a second).
>
> Just wondering which tcp parameter causes the long delay

No idea what tweaks you have made to your machine.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: postfix milter body chunk length

Matthias Schneider
In reply to this post by Wietse Venema
Hi Wietse,


> I suspect that the bottleneck is on the receiving side.
>
> - Maybe the loopback stack does not like the 65535 block size. Try
> using an ethernet interface address instead.

I tried switching from interface lo to eno1 - same issue.


>
> - Maybe the application "appends" the new content to the end of a
> large string; that requires allocating and copying huge amounts of
> memory each time. That alone may explain the difference in program
> speed.

I just append to a bytes buffer, its very fast on unix socket (same
code), its also fast when i use a own milterclient implementation in golang.


>
> - Maybe the above program behavior triggers a worst case in the
> malloc library as it needs to deal with copies of strings of
> increasing length.
>
> You can test that with a test program that does nothing with the
> data, such as the test-milter program that is part of Postfix
> source code.
>
> Wietse

I compared some tcpdumps with postfix as client and my milterclient
(which is fast) as client.

It seems that we run into nagle timeout (the ACK is always delayed).
Setting TCP_QUICKACK on my milter socket doesnt help, is there a way to
set TCP_NODELAY on the milter connection in postfix?


Matthias Schneider

Reply | Threaded
Open this post in threaded view
|

Re: postfix milter body chunk length

Wietse Venema
Matthias Schneider:
> Hi Wietse,
>
>
> > I suspect that the bottleneck is on the receiving side.
> >
> > - Maybe the loopback stack does not like the 65535 block size. Try
> > using an ethernet interface address instead.
>
> I tried switching from interface lo to eno1 - same issue.

What's the en01 MTU size? Postfix has code (in vstream_tweak.c
which is called by the Milter client and other Postfix code) to
automatically increase the VSTREAM buffer from 4096 to the maximal
segment size, to avoid Nagle delays, though that would not help if
the MTU is > 65535 (the Milter's 'packet' size).

I would therefore not expect Nagle delays on en01.

> I compared some tcpdumps with postfix as client and my milterclient
> (which is fast) as client.
>
> It seems that we run into nagle timeout (the ACK is always delayed).
> Setting TCP_QUICKACK on my milter socket doesnt help, is there a way to
> set TCP_NODELAY on the milter connection in postfix?

You've got the source.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: postfix milter body chunk length

Chris Wedgwood
In reply to this post by Matthias Schneider
On Mon, Aug 19, 2019 at 10:34:51AM +0200, Matthias Schneider wrote:
> Chris, can you tell me your postfix version/settings?

mail_version = 3.4.5
milter_protocol = 6

(not sure what other settings are relevant here)

> But postfix (3.3.0 and 3.4.5) only sends about 24 body chunks per
> second to my milter application. Its the only milter configured and
> the milter is running on 127.0.0.1 (so no latency issues)

i tested over a unix domain socket

i imagine this is faster than tcp but haven't checked
Reply | Threaded
Open this post in threaded view
|

Re: postfix milter body chunk length

Matthias Schneider
In reply to this post by Wietse Venema
I want to share my research about my milter tcp speed issue:


I have turned on smtpd -v in master.cf to see the mss in the logfile:

On my ubuntu 18.04 the loopback device has a mtu of 65536 which results
in a "vstream_tweak_tcp: TCP_MAXSEG 21845 "
This is slow and has the nagle timeout problem.

When I set the mtu to 65535 (or 65534) it still gets TCP_MAXSEG 21845
(and is still slow).


Once i do "ip link set dev lo mtu 21845" i will get vstream_tweak_tcp:
TCP_MAXSEG 21793 and the performance is great (only 1 second for 100mb
body).


Installing the Milter on another machine (which is available throught
the eno1 (1500mtu) interface i will get TCP_MAXSEG 1448 and still bad
performance (43secs instead of 1.30m).


Since this is strange i prefer to switch to unixsocket.


I also researched where the delay happened in my milter: its while
waiting for the 64k packet
https://github.com/mschneider82/milter/blob/b4a6d7c78ac39b8a1d71b53b74bc08d95652d310/session.go#L101


Best regards

Matthias Schneider



Am 19.08.19 um 20:05 schrieb Wietse Venema:

> Matthias Schneider:
>> Hi Wietse,
>>
>>
>>> I suspect that the bottleneck is on the receiving side.
>>>
>>> - Maybe the loopback stack does not like the 65535 block size. Try
>>> using an ethernet interface address instead.
>> I tried switching from interface lo to eno1 - same issue.
> What's the en01 MTU size? Postfix has code (in vstream_tweak.c
> which is called by the Milter client and other Postfix code) to
> automatically increase the VSTREAM buffer from 4096 to the maximal
> segment size, to avoid Nagle delays, though that would not help if
> the MTU is > 65535 (the Milter's 'packet' size).
>
> I would therefore not expect Nagle delays on en01.
>
>> I compared some tcpdumps with postfix as client and my milterclient
>> (which is fast) as client.
>>
>> It seems that we run into nagle timeout (the ACK is always delayed).
>> Setting TCP_QUICKACK on my milter socket doesnt help, is there a way to
>> set TCP_NODELAY on the milter connection in postfix?
> You've got the source.
>
> Wietse
Reply | Threaded
Open this post in threaded view
|

Re: postfix milter body chunk length

Wietse Venema
Matthias Schneider:
> I want to share my research about my milter tcp speed issue:
>
>
> I have turned on smtpd -v in master.cf to see the mss in the logfile:
>
> On my ubuntu 18.04 the loopback device has a mtu of 65536 which results
> in a "vstream_tweak_tcp: TCP_MAXSEG 21845 "
> This is slow and has the nagle timeout problem.

Why does the kernel return this nonsensical mss value which is
1/3 of the MTU? 40kB of packet the framing overhead for loopback?

> When I set the mtu to 65535 (or 65534) it still gets TCP_MAXSEG 21845
> (and is still slow).
>
>
> Once i do "ip link set dev lo mtu 21845" i will get vstream_tweak_tcp:
> TCP_MAXSEG 21793 and the performance is great (only 1 second for 100mb
> body).
>
>
> Installing the Milter on another machine (which is available throught
> the eno1 (1500mtu) interface i will get TCP_MAXSEG 1448 and still bad
> performance (43secs instead of 1.30m).

Are you on PPPoE or some other tunnel? I would expect mss=1460
for TCP/IP over ethernet.

> Since this is strange i prefer to switch to unixsocket.

Don't bother fighting with a hostile network stack :-)

        Wietse

>
> I also researched where the delay happened in my milter: its while
> waiting for the 64k packet
> https://github.com/mschneider82/milter/blob/b4a6d7c78ac39b8a1d71b53b74bc08d95652d310/session.go#L101
>
>
> Best regards
>
> Matthias Schneider
>
>
>
> Am 19.08.19 um 20:05 schrieb Wietse Venema:
> > Matthias Schneider:
> >> Hi Wietse,
> >>
> >>
> >>> I suspect that the bottleneck is on the receiving side.
> >>>
> >>> - Maybe the loopback stack does not like the 65535 block size. Try
> >>> using an ethernet interface address instead.
> >> I tried switching from interface lo to eno1 - same issue.
> > What's the en01 MTU size? Postfix has code (in vstream_tweak.c
> > which is called by the Milter client and other Postfix code) to
> > automatically increase the VSTREAM buffer from 4096 to the maximal
> > segment size, to avoid Nagle delays, though that would not help if
> > the MTU is > 65535 (the Milter's 'packet' size).
> >
> > I would therefore not expect Nagle delays on en01.
> >
> >> I compared some tcpdumps with postfix as client and my milterclient
> >> (which is fast) as client.
> >>
> >> It seems that we run into nagle timeout (the ACK is always delayed).
> >> Setting TCP_QUICKACK on my milter socket doesnt help, is there a way to
> >> set TCP_NODELAY on the milter connection in postfix?
> > You've got the source.
> >
> > Wietse
>
Reply | Threaded
Open this post in threaded view
|

Re: postfix milter body chunk length

Matthias Schneider
Am 20.08.19 um 13:09 schrieb Wietse Venema:
> Why does the kernel return this nonsensical mss value which is
> 1/3 of the MTU? 40kB of packet the framing overhead for loopback?


Good question, i have the same problem on different ubuntu machines,
there is no tunneling or vpn, just plain network.
I also get TCP_MAXSEG 21845 on a newly launched VPS on a cloud provider
(loopback mtu 65536).


net.ipv4.tcp_congestion_control = cubic

but i have tried "reno" too, no difference. I have no idea where the mtu
/ 3 happens.



Thanks

Matthias Schneider



> Are you on PPPoE or some other tunnel? I would expect mss=1460
> for TCP/IP over ethernet.
>
>> Since this is strange i prefer to switch to unixsocket.
> Don't bother fighting with a hostile network stack :-)
>
> Wietse
>> I also researched where the delay happened in my milter: its while
>> waiting for the 64k packet
>> https://protection.retarus.com/v1?u=https%3A%2F%2Fgithub.com%2Fmschneider82%2Fmilter%2Fblob%2Fb4a6d7c78ac39b8a1d71b53b74bc08d95652d310%2Fsession.go%23L101&c=3ii9Gwp&r=6kteDpYm1BWnNiGBD8pxt3&k=7s1&s=Qa2GzZMyAcLdZiwoBnfkEaAvdOpcAbRSJXPoLZJBjYB
>>
>>
>> Best regards
>>
>> Matthias Schneider
>>
>>
>>
>> Am 19.08.19 um 20:05 schrieb Wietse Venema:
>>> Matthias Schneider:
>>>> Hi Wietse,
>>>>
>>>>
>>>>> I suspect that the bottleneck is on the receiving side.
>>>>>
>>>>> - Maybe the loopback stack does not like the 65535 block size. Try
>>>>> using an ethernet interface address instead.
>>>> I tried switching from interface lo to eno1 - same issue.
>>> What's the en01 MTU size? Postfix has code (in vstream_tweak.c
>>> which is called by the Milter client and other Postfix code) to
>>> automatically increase the VSTREAM buffer from 4096 to the maximal
>>> segment size, to avoid Nagle delays, though that would not help if
>>> the MTU is > 65535 (the Milter's 'packet' size).
>>>
>>> I would therefore not expect Nagle delays on en01.
>>>
>>>> I compared some tcpdumps with postfix as client and my milterclient
>>>> (which is fast) as client.
>>>>
>>>> It seems that we run into nagle timeout (the ACK is always delayed).
>>>> Setting TCP_QUICKACK on my milter socket doesnt help, is there a way to
>>>> set TCP_NODELAY on the milter connection in postfix?
>>> You've got the source.
>>>
>>> Wietse
12