Postfix performance problem (cleanup process)

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Postfix performance problem (cleanup process)

Emanuel
Hello everyone, I'm representing a performance problem on my server.

I explain in detail the configuration of my server.

I am using postfix with 46 IPs configured as a mta, with round-robin, in
the master.cf file

I think the "cleanup" process is responsible for the excessive use of cpu.

ps fax | grep -c cleanup
181

Is there any way to limit it?

I have modified the maxproc in the master.cf file but the change was not
made.

# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (yes)   (never) (100)
# ==========================================================================

smht-101-76       unix  -       -       n       -       10 smtp
#  -o syslog_name=smht-101-76
   -o smtp_helo_name=smht-101-76.domain.com
   -o smtp_bind_address=x.x.x.x
x.x.x.x:25  inet  n       -       n       -     20       smtpd

==> per mta maxproc is declared in 10
# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (yes)   (never) (100)
# ==========================================================================
#628       inet  n       -       n       -       -       qmqpd
pickup    unix  n       -       n       60      1       pickup
cleanup   unix  n       -       n       -       100       cleanup
qmgr      unix  n       -       n       300     1       qmgr
#qmgr      unix  n       -       n       300     1       oqmgr
tlsmgr    unix  -       -       n       1000?   1       tlsmgr
rewrite   unix  -       -       n       -       - trivial-rewrite
bounce    unix  -       -       n       -       0       bounce
defer     unix  -       -       n       -       0       bounce
trace     unix  -       -       n       -       0       bounce
verify    unix  -       -       n       -       1       verify
flush     unix  n       -       n       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
smtp      unix  -       -       n       -       -       smtp
relay     unix  -       -       n       -       -       smtp
#       -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
showq     unix  n       -       n       -       -       showq
error     unix  -       -       n       -       -       error
retry     unix  -       -       n       -       -       error
discard   unix  -       -       n       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       n       -       -       lmtp
anvil     unix  -       -       n       -       1       anvil
scache    unix  -       -       n       -       1       scache

hardware:

Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz

free -m
               total        used        free      shared buff/cache  
available
Mem:          31900       18626        5989        1658 7284       11102
Swap:          7628           0        7628


any sugestions:



--
Reply | Threaded
Open this post in threaded view
|

Re: Postfix performance problem (cleanup process)

Matus UHLAR - fantomas
On 18.04.18 11:00, Emanuel wrote:
>Hello everyone, I'm representing a performance problem on my server.
>
>I explain in detail the configuration of my server.
>
>I am using postfix with 46 IPs configured as a mta, with round-robin,
>in the master.cf file

46 IPs? why?

>I think the "cleanup" process is responsible for the excessive use of cpu.
>
>ps fax | grep -c cleanup
>181

the whole fact that there's 181 instances of a process does not mean they
are eating your CPU.

"top" could give you more information.

However, the question is: how many mails does the machine process?


>I have modified the maxproc in the master.cf file but the change was
>not made.

have you reloaded postfix by issuing "postfix reload"?


>free -m
>              total        used        free      shared buff/cache  
>available
>Mem:          31900       18626        5989        1658 7284       11102
>Swap:          7628           0        7628

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Quantum mechanics: The dreams stuff is made of.
Reply | Threaded
Open this post in threaded view
|

Re: Postfix performance problem (cleanup process)

Emanuel



El 18/04/18 a las 12:03, Matus UHLAR - fantomas escribió:
On 18.04.18 11:00, Emanuel wrote:
Hello everyone, I'm representing a performance problem on my server.

I explain in detail the configuration of my server.

I am using postfix with 46 IPs configured as a mta, with round-robin, in the master.cf file

46 IPs? why?
thank you for responding, we send transactional mail, we have around 100,000 clients with their respective email accounts.

I think the "cleanup" process is responsible for the excessive use of cpu.

ps fax | grep -c cleanup
181

the whole fact that there's 181 instances of a process does not mean they
are eating your CPU.

"top" could give you more information.

However, the question is: how many mails does the machine process?

top - 12:10:14 up 5 days, 17:06,  4 users,  load average: 13,32, 17,52, 18,43
Tasks: 680 total,   3 running, 677 sleeping,   0 stopped,   0 zombie
%Cpu0  :  9,8 us,  3,0 sy,  0,0 ni, 75,3 id,  7,1 wa,  0,0 hi,  4,7 si,  0,0 st
%Cpu1  :  7,0 us,  2,3 sy,  0,0 ni, 89,6 id,  1,0 wa,  0,0 hi,  0,0 si,  0,0 st
%Cpu2  :  3,0 us,  2,7 sy,  0,0 ni, 88,6 id,  5,7 wa,  0,0 hi,  0,0 si,  0,0 st
%Cpu3  :  5,0 us,  2,0 sy,  0,0 ni, 92,4 id,  0,3 wa,  0,0 hi,  0,3 si,  0,0 st
%Cpu4  :  4,7 us,  1,7 sy,  0,0 ni, 93,4 id,  0,3 wa,  0,0 hi,  0,0 si,  0,0 st
%Cpu5  :  2,0 us,  1,7 sy,  0,0 ni, 96,0 id,  0,3 wa,  0,0 hi,  0,0 si,  0,0 st
%Cpu6  :  3,7 us,  1,3 sy,  0,0 ni, 94,4 id,  0,7 wa,  0,0 hi,  0,0 si,  0,0 st
%Cpu7  :  4,3 us,  1,3 sy,  0,0 ni, 94,3 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
KiB Mem : 32665920 total, 11548712 free, 13041936 used,  8075272 buff/cache
KiB Swap:  7812092 total,  7811228 free,      864 used. 17244680 avail Mem

 Traffic ------------- active/incoming connections ---------------------
                                        T   5 10 20 40 80 160 320 640 1280 1280+
                                TOTAL 131 131  0  0  0  0   0   0   0    0     0
                            gmail.com  20  20  0  0  0  0   0   0   0    0     0
                          hotmail.com  12  12  0  0  0  0   0   0   0    0     0
                         yahoo.com.ar   4   4  0  0  0  0   0   0   0    0     0
                      cathodicpro.com   4   4  0  0  0  0   0   0   0    0     0
                    cannata-bruno.com   4   4  0  0  0  0   0   0   0    0     0

 Traffic ------------- DEFERRED connections  ---------------------------
                                        T  5 10 20 40  80 160 320 640 1280 1280+
                                TOTAL 781 42 38 68 80 159 211 180   3    0     0
                            gmail.com 173  3  6 12 18  26  55  52   1    0     0
                      arnetbiz.com.ar  12  1  2  5  4   0   0   0   0    0     0
                guidesolutions.com.ar  12  1  0  1  1   2   2   3   2    0     0
                     fastdieta.review  11  0  0  0  0   0   5   6   0    0     0
              jerarquicossalud.com.ar  11  0  0  0  0   1   7   3   0    0     0

 Traffic ------------- DEFERRED sent connections  ----------------------
                                        T  5 10 20 40  80 160 320 640 1280 1280+




I have modified the maxproc in the master.cf file but the change was not made.

have you reloaded postfix by issuing "postfix reload"?


free -m
              total        used        free      shared buff/cache   available
Mem:          31900       18626        5989        1658 7284       11102
Swap:          7628           0        7628


--
envialosimple.com
Emanuel Gonzalez
IT / Departamento Emails
[hidden email]
www.envialosimple.com
by donweb
 
Nota de confidencialidad: Este mensaje y archivos adjuntos al mismo son confidenciales, de uso exclusivo para el destinatario del mismo. La divulgación y/o uso del mismo sin autorización por parte de DonWeb.com queda prohibida.
DonWeb.com no se hace responsable del mensaje por la falsificación y/o alteración del mismo.
De no ser Ud el destinatario del mismo y lo ha recibido por error, por favor, notifique al remitente y elimínelo de su sistema.
Confidentiality Note: This message and any attachments (the message) are confidential and intended solely for the addressees. Any unauthorised use or dissemination is prohibited by DonWeb.com.
DonWeb.com shall not be liable  for the message if altered or falsified.
If you are not the intended addressee of this message, please cancel it immediately and inform the sender
Nota de Confidencialidade: Esta mensagem e seus eventuais anexos podem conter dados confidenciais ou privilegiados.
Se você os recebeu por engano ou não é um dos destinatários aos quais ela foi endereçada, por favor destrua-a e a todos os seus eventuais anexos ou copias realizadas, imediatamente.
É proibida a retenção, distribuição, divulgação ou utilização de quaisquer informações aqui contidas.
Por favor, informenos sobre o recebimento indevido desta mensagem, retornando-a para o autor.
 
Reply | Threaded
Open this post in threaded view
|

Re: Postfix performance problem (cleanup process)

Wietse Venema
In reply to this post by Emanuel
Emanuel:
[ Charset windows-1252 converted... ]

> Hello everyone, I'm representing a performance problem on my server.
>
> I explain in detail the configuration of my server.
>
> I am using postfix with 46 IPs configured as a mta, with round-robin, in
> the master.cf file
>
> I think the "cleanup" process is responsible for the excessive use of cpu.
>
> ps fax | grep -c cleanup
> 181
>
> Is there any way to limit it?

Limit the number of processes? No. There is one per SMTP daemon.

Limit the CPU per process? Fewer header/body checks.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Your HSBC application documents

Security

HSBC Spearphishing Campaign Sinkhole

You may have received an email with the title "FW: Your HSBC application documents" by [hidden email] or [hidden email]. If you have done so, then this was not an legitimate email by HSBC bank and instead was what's known as an "phishing email" with the intention to trick you into clicking malicious files that may steal your account information and/or infect your computer with an virus.

I have purchased this domain after it being terminated by the hosting provider to prevent further users from being scammed. (NOTE: I am not responsible for those emails and/or any malicious cyber attacks. If HSBC are reading this email, then please get in contact for further information: [hidden email]).

If you have opened the "04172018HSBCJSZZH_app.doc" file, then I recommend scanning your computer for viruses as it may be infected, as well as contacting your local law enforcement agency.

United Kingdom

Action Fruad - https://www.actionfraud.police.uk/report_fraud

National Cyber Security Center https://www.ncsc.gov.uk/

United States of America

FBI - https://www.ic3.gov/default.aspx

Further Information:

https://www.ncsc.gov.uk/phishing

https://www.fbi.gov/scams-and-safety/on-the-internet

Regards,

James Hemmings

Cyber Security Researcher

[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Postfix performance problem (cleanup process)

Emanuel
In reply to this post by Wietse Venema

Can the files header/body_checks generate overload?

Postfix configuration:

default_process_limit = 300

smtpd_client_connection_count_limit = 2000

smtpd_recipient_limit = 20

Any information you give me is helpful.


El 18/04/18 a las 14:00, Wietse Venema escribió:
Limit the number of processes? No. There is one per SMTP daemon.

Limit the CPU per process? Fewer header/body checks.

--
envialosimple.com
Emanuel Gonzalez
IT / Departamento Emails
[hidden email]
www.envialosimple.com
by donweb
 
Nota de confidencialidad: Este mensaje y archivos adjuntos al mismo son confidenciales, de uso exclusivo para el destinatario del mismo. La divulgación y/o uso del mismo sin autorización por parte de DonWeb.com queda prohibida.
DonWeb.com no se hace responsable del mensaje por la falsificación y/o alteración del mismo.
De no ser Ud el destinatario del mismo y lo ha recibido por error, por favor, notifique al remitente y elimínelo de su sistema.
Confidentiality Note: This message and any attachments (the message) are confidential and intended solely for the addressees. Any unauthorised use or dissemination is prohibited by DonWeb.com.
DonWeb.com shall not be liable  for the message if altered or falsified.
If you are not the intended addressee of this message, please cancel it immediately and inform the sender
Nota de Confidencialidade: Esta mensagem e seus eventuais anexos podem conter dados confidenciais ou privilegiados.
Se você os recebeu por engano ou não é um dos destinatários aos quais ela foi endereçada, por favor destrua-a e a todos os seus eventuais anexos ou copias realizadas, imediatamente.
É proibida a retenção, distribuição, divulgação ou utilização de quaisquer informações aqui contidas.
Por favor, informenos sobre o recebimento indevido desta mensagem, retornando-a para o autor.
 
Reply | Threaded
Open this post in threaded view
|

Re: Postfix performance problem (cleanup process)

Wietse Venema
Emanuel:
> Can the files header/body_checks generate overload?

I have not seen evidence that the cleanup daemon is responsible for
a performance issue. Absent meaningful info, no support.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Postfix performance problem (cleanup process)

Viktor Dukhovni
In reply to this post by Emanuel


> On Apr 18, 2018, at 2:51 PM, Emanuel <[hidden email]> wrote:
>
> smtpd_recipient_limit = 20

RFC5321 requires at least 100.  Unnecessarily splitting the envelope does not help performance.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: Postfix performance problem (cleanup process)

Micah Anderson-2
In reply to this post by Emanuel
Emanuel <[hidden email]> writes:

> Can the files header/body_checks generate overload?

Yes.

I recently tried to load the malware patrol header check list, which has
49349 lines as a regexp body check, and I quickly had to stop doing that
because the resource usage of the machine quickly went into a state
where it was no longer stable, causing heavy CPU usage in the cleanup
processes and the OOM killer to get crazy with the cheeze-whiz.

micah
Reply | Threaded
Open this post in threaded view
|

Re: Postfix performance problem (cleanup process)

Peer Heinlein
In reply to this post by Emanuel
Am 18.04.2018 um 20:51 schrieb Emanuel:
> Can the files header/body_checks generate overload?

If you have many header/body_checks you'll see a higher usage of the
cleanup process in tools like "top".

You can save a lot of cpu ressources if you use...

...pcre instead of regexp (mostly the syntax is the same, but the engine
is better, just change the prefix!)

...use "if..." constructs to group

from
sender
subject

patterns in header checks

...if you use better one line with a more generic/complex pattern
instead of using multiple very specific lines with just slightly
different pattern.

Peer

Reply | Threaded
Open this post in threaded view
|

Re: Postfix performance problem (cleanup process)

Dominic Raferd
On 19 April 2018 at 07:21, Peer Heinlein <[hidden email]> wrote:
>
> You can save a lot of cpu ressources if you use...
>
> ...pcre instead of regexp (mostly the syntax is the same, but the engine
> is better, just change the prefix!)

Check if supported with:
# postconf -m|grep pcre
pcre

If using the standard postfix under Debian (etc), add pcre support with:
# apt-get install postfix-pcre
Reply | Threaded
Open this post in threaded view
|

Re: Postfix performance problem (cleanup process)

Matus UHLAR - fantomas
In reply to this post by Micah Anderson-2
>Emanuel <[hidden email]> writes:
>> Can the files header/body_checks generate overload?

On 19.04.18 00:05, micah wrote:
>Yes.
>
>I recently tried to load the malware patrol header check list, which has
>49349 lines as a regexp body check, and I quickly had to stop doing that
>because the resource usage of the machine quickly went into a state
>where it was no longer stable, causing heavy CPU usage in the cleanup
>processes and the OOM killer to get crazy with the cheeze-whiz.

this is true, however it should cause high CPU usage, the OP has shown
nearly none.  both memory and CPU were not overloaded.

turning them off is still worth trying.
--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Support bacteria - they're the only culture some people have.