smtpd - high memory usage

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

smtpd - high memory usage

Mantas Mikulėnas
Hello,

I have a hobby server that does a little bit of everything, including
1) receiving email via Postfix as a backup MX,
2) receiving ~70k IPv6 routes via BGP.

The problem I'm having is that when all ~70k routes are loaded into
the kernel (Linux), this somehow causes high memory usage in Postfix
"smtpd" processes -- as soon as the first client connects, I get a
smtpd process that's around ~130 MB (compared to the more usual ~13 MB
when BGP is down). This even occurs if it's an IPv4 client.

I am trying to reduce that a bit, but I could not find any options in
postconf that would be related to IP routes (except for
mynetworks/mynetworks_style, but configuring it manually did not
really help). I couldn't even find anything in the source code that
would be routing-related, either.

(I'm using Linux. The routes are in table 1, *not* in the "main" table.)

# postconf -nf
alias_maps = hash:/etc/aliases
compatibility_level = 2
default_process_limit = 10
mydomain = nullroute.eu.org
mynetworks = 127.0.0.1/32, [::1]/128
myorigin = $mydomain
relay_domains = $mydestination, nullroute.eu.org
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtp_tls_cert_file = /etc/private/host.crt
smtp_tls_key_file = /etc/private/host.key
smtp_tls_loglevel = 1
smtp_tls_security_level = may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_client_connection_count_limit = 3
smtpd_client_port_logging = yes
smtpd_client_restrictions = permit_mynetworks reject_rbl_client zen.spamhaus.org
smtpd_recipient_restrictions = reject_rbl_client zen.spamhaus.org
    check_policy_service unix:private/policyd-spf
smtpd_relay_restrictions = permit_mynetworks permit_tls_clientcerts
    reject_unauth_destination
smtpd_tls_ask_ccert = yes
smtpd_tls_cert_file = /etc/letsencrypt/live/$myhostname/fullchain.pem
smtpd_tls_dh1024_param_file = /etc/ssl/dh4096.pem
smtpd_tls_fingerprint_digest = sha1
smtpd_tls_key_file = /etc/letsencrypt/live/$myhostname/privkey.pem
smtpd_tls_loglevel = 1
smtpd_tls_received_header = yes
smtpd_tls_security_level = may


--
Mantas Mikulėnas
Reply | Threaded
Open this post in threaded view
|

Re: smtpd - high memory usage

Mantas Mikulėnas
Disregard this; I'm sure I have found the nss module that causes high
memory usage.

--
Mantas Mikulėnas
Reply | Threaded
Open this post in threaded view
|

Re: smtpd - high memory usage

Wietse Venema
Mantas Mikul?nas:
> Disregard this; I'm sure I have found the nss module that causes high
> memory usage.

Which nss module would that be? Postfix does not care how many routes
a system has, but it does care about local network interface addresses
and netmasks if main.cf:mynetworks does not specify those explicitly.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: smtpd - high memory usage

Mantas Mikulėnas
On Sat, Jul 27, 2019, 22:35 Wietse Venema <[hidden email]> wrote:
>
> Mantas Mikul?nas:
> > Disregard this; I'm sure I have found the nss module that causes high
> > memory usage.
>
> Which nss module would that be?

It was libnss_myhostname (specifically the version that is included
with systemd).

Besides handling the system hostname as advertised, the
systemd-bundled variant also recognizes the magic hostname "_gateway"
and resolves it to wherever the OS default routes point, and the other
way around. So whenever Postfix does a reverse-DNS lookup and the
regular "files"/"dns" modules yield no result, the "myhostname" module
scans all routing tables to compare the query against the default
gateways, leaking a bit of memory after doing so.

> Postfix does not care how many routes
> a system has, but it does care about local network interface addresses
> and netmasks if main.cf:mynetworks does not specify those explicitly.
>
>         Wietse
Reply | Threaded
Open this post in threaded view
|

Re: smtpd - high memory usage

Viktor Dukhovni
On Sun, Jul 28, 2019 at 12:05:33AM +0300, Mantas Mikulėnas wrote:

> On Sat, Jul 27, 2019, 22:35 Wietse Venema <[hidden email]> wrote:
> >
> > Mantas Mikul?nas:
> > > Disregard this; I'm sure I have found the nss module that causes high
> > > memory usage.
> >
> > Which nss module would that be?
>
> It was libnss_myhostname (specifically the version that is included
> with systemd).
>
> Besides handling the system hostname as advertised, the
> systemd-bundled variant also recognizes the magic hostname "_gateway"
> and resolves it to wherever the OS default routes point, and the other
> way around. So whenever Postfix does a reverse-DNS lookup and the
> regular "files"/"dns" modules yield no result, the "myhostname" module
> scans all routing tables to compare the query against the default
> gateways, leaking a bit of memory after doing so.

Wow!  Though lately I'm inclined to be more sympathetic to the
*goals* of systemd, the continuing implementation warts and adverse
performance impact make it difficult to support its lofty causes.

The commercial Linux vendors really need to put more muscle behind
making systemd reliable and performant, it is a critical component
of the OS, and needs to be better implemented.

Presently, at least logging and name resolution are in sad shape
on many Linux servers.

--
        Viktor.