Connection Caching for One Transport

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Connection Caching for One Transport

Greg Sims
We are distributing a daily email to our subscribers -- which generates a large burst of email. We have main.cf configured as follows:

sender_dependent_default_transport_maps = randmap:{r192,r193,r194,r195,r196,r197,r198}
smtp_connection_cache_on_demand=no

Distributing the traffic across seven ip addresses is working well for all but one major transport.

This one majortransport is complaining about too many connections even with: 

majortransport_destination_concurrency_limit = 4

I am configuring majortransport to use a single ip address in master.cf to reduce the number of connections.  I hope this will allow an increased destination_concurrency_limit and better throughput for majortransport.

This seems like a good scenario to use  smtp_connection_cache_on_demand=yes.  Is it possible to configure this for just majortransport?  It seems turning on the connection_cache for the entire installation will reduce the randomness and effectiveness of the randmap.

Thanks, Greg
Reply | Threaded
Open this post in threaded view
|

Re: Connection Caching for One Transport

Wietse Venema
Greg Sims:
> This seems like a good scenario to
> use  smtp_connection_cache_on_demand=yes.  Is it possible to configure this
> for just majortransport?  It seems turning on the connection_cache for the
> entire installation will reduce the randomness and effectiveness of the
> randmap.

To override the main.cf option for specific SMTP clients, use master.cf

majortransport .. .. .. .. .. .. smtp
     -o smtp_connection_cache_on_demand=...

and "postfix reload".

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Connection Caching for One Transport

Viktor Dukhovni
In reply to this post by Greg Sims
On Wed, Jul 22, 2020 at 07:38:52AM -0700, Greg Sims wrote:

> We have main.cf configured as follows:
>
> sender_dependent_default_transport_maps =
> randmap:{r192,r193,r194,r195,r196,r197,r198}
> smtp_connection_cache_on_demand=no

Can you share your reason for avoiding connection caching in general?

> Distributing the traffic across seven ip addresses is working well for
> all but one major transport.

What do you mean by "one major transport"?  Do you mean "one major
destination"?  Do you have a custom transport configured for that
destination?

> This one major transport is complaining about too many connections
> even with:

Transports don't complain about too many connections, but a set of
receiving servers might.  Do these servers support lots of destination
domains?

One limitation of the way that Postfix handles destination concurrency
limits, is that these limits group messages by *recipient domain*, not
MX host.

A set of MX hosts that support many different domains are then
potentially handling concurrent connections for each domain, up to the
transport process limit, rather than the destination concurrency limit.

> majortransport_destination_concurrency_limit = 4

When you map a large provider's domain (say gmail.com) to a dedicated
transport, they might still see additional connections for all the
customer domains hosted by the same inbound MX hosts.

> I am configuring majortransport to use a single ip address in
> master.cf to reduce the number of connections.

Using but a single IP address increases the concurrent connects
per address.  On the contrary, you'd want to spread the traffic
over multiple IPs, assuming that they don't aggregate connections
across netblocks large enough to encompass your entire IP pool.

> I hope this will allow an increased destination_concurrency_limit and
> better throughput for majortransport.

I don't see how it would help.

> This seems like a good scenario to use
> smtp_connection_cache_on_demand=yes.

This does not reduce connection concurrency, the main effect is to
reduce connection setup latency, and only if you don't use TLS,
otherwise you'd also want to enable use of tlsproxy(8) connection reuse.
So it may increase throughput, but at the cost of slightly higher
concurrency, since some of the connections may be idle at any give
time.

> Is it possible to configure this for just majortransport?

Yes, but you're unlikely to benefit from this if your goal
is reducing connection concurrency.

> It seems turning on the connection_cache for the entire installation
> will reduce the randomness and effectiveness of the randmap.

No, because the transport is selected first, and connection caching
never mixes transports.  So once a message is assigned to a given
transport, it will only reuse connections cached for that transport.

Bottom line, you're struggling to think clearly about your problem,
and you're not quite clear on what the difference is between a
destination and transport, what destination concurrency means, ...

Read through more of the documentation:

    TUNING_README
    QSHAPE_README
    qmgr(8)

Read the Nostarch press Postfix book by Patrick and Ralf, ...

If you need to avoid hitting concurrency limits at a large cloud
provider hosting many recipient domains, you may need to channel the
traffic to a dedicated Postfix instance that handles only traffic for
that provider, and has a sufficiently low *process* limit for the
smtp(8) transport.

That can be done by censoring the DNS replies for the IP addresses of
the MX hosts in question and replacing them with your own IPs that
direct traffic to the dedicated Postfix instance that handles traffic to
that provider.

Postfix would need a substantially modified queue manager to directly
limit traffic concurrency by destination IP address.

--
    Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Connection Caching for One Transport

Greg Sims
> > We have main.cf configured as follows:
> >
> > sender_dependent_default_transport_maps =
> > randmap:{r192,r193,r194,r195,r196,r197,r198}
> > smtp_connection_cache_on_demand=no
>
> Can you share your reason for avoiding connection caching in general?

Our current mail server uses one ip address and it running so slow that we
can not deliver daily email to our subscribers in a timely fashion.  I found
this strategy of using multiple ip addresses here:


The blog states that connection_cache=no will "Prevent the SMTP clients 
from caching connections, so sending multiple emails to a single domain 
does not end up using the same SMTP client."  I interpreted this as 
connection_cache=no will increase the random effect of using this strategy.  
We are now seeing 30 times the throughput of our previous mail server. This
is huge step in the right direction.

One major destination (outlook.com) is complaining about the number of 
connections it is seeing using this strategy.  This destination is running on a 
dedicated transport in master.cf that only specifies:

  outlook  unix  -       -       n       -       -       smtp
    -o syslog_name=outlook

transport.regexp has seven domains that use the "outlook:" transport. 

main.cf contains:

  outlook_initial_destination_concurrency = 4
  outlook_destination_concurrency_limit = 4
  outlook_destination_recipient_limit = 2
  outlook_destination_rate_delay = 0s
  outlook_destination_concurrency_failed_cohort_limit = 10
  outlook_destination_concurrency_positive_feedback = 1/3
  outlook_destination_concurrency_negative_feedback = 1/8

What is the best way to configure for the following message from outlook.com
in the maillog:

  said: 451 4.7.652 The mail server [<source_server_ip>] has exceeded the 
  maximum number of connections.

Please note the email is being created on a VM with <source_server_ip>.
The email is then sent to our new mail server via SMTP.  It appears outlook
is complaining about the number of connections for email coming from the
source_server_ip -- only one ip address.

Possible solutions include:
  (1) reduce outlook_destination_concurrency_limit = 4 with a minimum of 2
  (2) send emails from multiple ip addresses on source_server (lots of work!)
  (3) use only one IP on the mail server for outlook transport (I agree, likely no help)
  (4) other ideas?

Thanks, Greg


On Wed, Jul 22, 2020 at 8:37 AM Viktor Dukhovni <[hidden email]> wrote:
On Wed, Jul 22, 2020 at 07:38:52AM -0700, Greg Sims wrote:

> We have main.cf configured as follows:
>
> sender_dependent_default_transport_maps =
> randmap:{r192,r193,r194,r195,r196,r197,r198}
> smtp_connection_cache_on_demand=no

Can you share your reason for avoiding connection caching in general?

> Distributing the traffic across seven ip addresses is working well for
> all but one major transport.

What do you mean by "one major transport"?  Do you mean "one major
destination"?  Do you have a custom transport configured for that
destination?

> This one major transport is complaining about too many connections
> even with:

Transports don't complain about too many connections, but a set of
receiving servers might.  Do these servers support lots of destination
domains?

One limitation of the way that Postfix handles destination concurrency
limits, is that these limits group messages by *recipient domain*, not
MX host.

A set of MX hosts that support many different domains are then
potentially handling concurrent connections for each domain, up to the
transport process limit, rather than the destination concurrency limit.

> majortransport_destination_concurrency_limit = 4

When you map a large provider's domain (say gmail.com) to a dedicated
transport, they might still see additional connections for all the
customer domains hosted by the same inbound MX hosts.

> I am configuring majortransport to use a single ip address in
> master.cf to reduce the number of connections.

Using but a single IP address increases the concurrent connects
per address.  On the contrary, you'd want to spread the traffic
over multiple IPs, assuming that they don't aggregate connections
across netblocks large enough to encompass your entire IP pool.

> I hope this will allow an increased destination_concurrency_limit and
> better throughput for majortransport.

I don't see how it would help.

> This seems like a good scenario to use
> smtp_connection_cache_on_demand=yes.

This does not reduce connection concurrency, the main effect is to
reduce connection setup latency, and only if you don't use TLS,
otherwise you'd also want to enable use of tlsproxy(8) connection reuse.
So it may increase throughput, but at the cost of slightly higher
concurrency, since some of the connections may be idle at any give
time.

> Is it possible to configure this for just majortransport?

Yes, but you're unlikely to benefit from this if your goal
is reducing connection concurrency.

> It seems turning on the connection_cache for the entire installation
> will reduce the randomness and effectiveness of the randmap.

No, because the transport is selected first, and connection caching
never mixes transports.  So once a message is assigned to a given
transport, it will only reuse connections cached for that transport.

Bottom line, you're struggling to think clearly about your problem,
and you're not quite clear on what the difference is between a
destination and transport, what destination concurrency means, ...

Read through more of the documentation:

    TUNING_README
    QSHAPE_README
    qmgr(8)

Read the Nostarch press Postfix book by Patrick and Ralf, ...

If you need to avoid hitting concurrency limits at a large cloud
provider hosting many recipient domains, you may need to channel the
traffic to a dedicated Postfix instance that handles only traffic for
that provider, and has a sufficiently low *process* limit for the
smtp(8) transport.

That can be done by censoring the DNS replies for the IP addresses of
the MX hosts in question and replacing them with your own IPs that
direct traffic to the dedicated Postfix instance that handles traffic to
that provider.

Postfix would need a substantially modified queue manager to directly
limit traffic concurrency by destination IP address.

--
    Viktor.