Outgoing SMTP throttling by IP

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Outgoing SMTP throttling by IP

Roberto Ragusa
Hi,

is there a way to throttle outgoing SMTP by destination IP?

My problem is that I send mails to
- domain1.com
- domain2.com
- domain3.com
which handle mails through
- mx.domain1.com
- mx.domain2.com
- mx.domain3.com
but when I start contacting them they easily complain with
"too many concurrent connections" because all the mx hosts
have been resolved to the same IP (well, IP pool, actually).
These domains (not under my control) are hosted on a provider
that has shared SMTP servers for all their customers and
I do not even have an easy way to enumerate the domains.

Can I let postfix realize that the connections are not
independent and that I want a parallelism limit for them?

Of course I want to keep high parallelism for all the other
normal domains.

Thanks.

--
    Roberto Ragusa    mail at robertoragusa.it
Reply | Threaded
Open this post in threaded view
|

Re: Outgoing SMTP throttling by IP

Wietse Venema
Roberto Ragusa:
> Hi,
>
> is there a way to throttle outgoing SMTP by destination IP?

No. The Postfix scheduler does not know about IP addresses. That
is a very fundamental property of the design. It schedules deliveries
in parallel, based on domain names.

Otherwise, if DNS lookups for one domain are slow, all other mail
deliveries would have to wait until Postfix has figured out the IP
addresses for that slow domain.

> My problem is that I send mails to
> - domain1.com
> - domain2.com
> - domain3.com
> which handle mails through
> - mx.domain1.com
> - mx.domain2.com
> - mx.domain3.com
> but when I start contacting them they easily complain with
> "too many concurrent connections" because all the mx hosts
> have been resolved to the same IP (well, IP pool, actually).
> These domains (not under my control) are hosted on a provider
> that has shared SMTP servers for all their customers and
> I do not even have an easy way to enumerate the domains.
>
> Can I let postfix realize that the connections are not
> independent and that I want a parallelism limit for them?

You could create transport_maps entries based on Postfix logging.
Basically, use transport_maps to direct all hosted domains to a
shared Postfix delivery transport with a low process limit.
All you need is a pattern matcher for each mail hosting provider
that is giving you trouble.

    Jun 17 10:28:54 spike postfix/smtp[81358]: 49n6rQ0RbkzJrNw:
    to=<[hidden email]>,
    relay=blah.provider.net[xxxx:xx00:0:1::3]:25, delay=0.44,
    delays=0.13/0.012/0.27/0.024, dsn=2.0.0, status=sent (250 2.0.0
    Ok: queued as 720A333A3B9)

Basically, look at the recipient domain in to= and the mail server
domain in relay=, that should be enough to decide that a recipient
domain is hosted with a known hosting provider.

This could use some autonation.

If you get fancy, you could use one dedicated transport per major
mail hosting provider, with its own process limit.

I would advise against using use transport_maps to override MX lookups.

        Wietse

> Of course I want to keep high parallelism for all the other
> normal domains.
>
> Thanks.
>
> --
>     Roberto Ragusa    mail at robertoragusa.it
>
Reply | Threaded
Open this post in threaded view
|

Re: Outgoing SMTP throttling by IP

@lbutlr
In reply to this post by Roberto Ragusa
On 17 Jun 2020, at 11:07, Roberto Ragusa <[hidden email]> wrote:
> but when I start contacting them they easily complain with "too many concurrent connections" because all the mx hosts have been resolved to the same IP (well, IP pool, actually). These domains (not under my control) are hosted on a provider that has shared SMTP servers for all their customers and I do not even have an easy way to enumerate the domains.

This sounds like a badly configured host, and I imagine they have to do a lot of work to allow large mailers (like google or outlook) to bypass there configuration in order for their customers to receive mail. They are probably setting a limit of a single connection for any server they have not excluded specifically. They know they are elating mail by doing this, they just don’t care.

I would try contacting them and if that fails (likely), add a footer to emails with something like “if your hosting provider is acme hosting LLC, your mail is probably severely delayed due to their misconfiguration of their server” and a link to a page explaining the issue. Since complaints from paying customers will certainly have weight if they ignore your query, this is much more likely to end up with a resolution.

Or you are sending a LOT of mail to them (but I bet if you test, they are limiting to a single connection). I’ve run into this behavior in the past when I hosted several mailman lists, the only solution was to acknowledge that some users would have their list mail delayed by hours (or sometimes a day or two) and when they complained show them why.