virtual transport lmtp vs. dovecot lda?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

virtual transport lmtp vs. dovecot lda?

David Mehler
Hello,

I'm running a Postfix 3.1 setup with Dovecot 2.29 and Mysql 5.7. I am
trying to track down an elusive problem. Previously I had my
virtual_transport set to dovecot with a dovecot service in master.cf.
I then enabled the lmtp service which uses a socket
/var/spool/postfix/private/dovecot-lmtp

I keep getting the error in the logs to many connections to the mysql
database and stuff is deferred.

Any ideas?

Thanks.
Dave.
Reply | Threaded
Open this post in threaded view
|

Re: virtual transport lmtp vs. dovecot lda?

Viktor Dukhovni

> On May 2, 2017, at 6:17 PM, David Mehler <[hidden email]> wrote:
>
> I keep getting the error in the logs to many connections to the mysql
> database and stuff is deferred.
>
> Any ideas?

Nothing specific, while you remain reticent about sharing the actual log
entries and your server configuration.  Generally, use "proxy:mysql:"
whenever you're otherwise tempted to use "mysql:".

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: virtual transport lmtp vs. dovecot lda?

David Mehler
Hi,

I'm not sure what to send. I've temporarily solved the problem by
increasing the mysql max_connections setting from 256 to 300 and
started working. Something is using up mysql processes when the lmtp
socket is used.

Dave.


On 5/2/17, Viktor Dukhovni <[hidden email]> wrote:

>
>> On May 2, 2017, at 6:17 PM, David Mehler <[hidden email]> wrote:
>>
>> I keep getting the error in the logs to many connections to the mysql
>> database and stuff is deferred.
>>
>> Any ideas?
>
> Nothing specific, while you remain reticent about sharing the actual log
> entries and your server configuration.  Generally, use "proxy:mysql:"
> whenever you're otherwise tempted to use "mysql:".
>
> --
> Viktor.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: virtual transport lmtp vs. dovecot lda?

Noel Jones-2
On 5/2/2017 6:33 PM, David Mehler wrote:
> Hi,
>
> I'm not sure what to send. I've temporarily solved the problem by
> increasing the mysql max_connections setting from 256 to 300 and
> started working. Something is using up mysql processes when the lmtp
> socket is used.
>
> Dave.

Postfix makes lots of mysql connections. This is normal and expected.

As Viktor already suggested, the solution is to use the postfix
proxymap service to consolidate the mysql connections. This will
greatly reduce the number of connections postfix makes to mysql and
usually improve performance too.

Documentation for proxymap is here:
http://www.postfix.org/proxymap.8.html
http://www.postfix.org/postconf.5.html#proxy_read_maps

Using the proxymap service is really easy. Generally, everywhere in
main.cf you have mysql:...  you replace with proxy:mysql:...
In some cases you may need to alter the default value of
proxy_read_maps.  There will be warnings in the log to guide you if
this is needed.



  -- Noel Jones

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

Reply | Threaded
Open this post in threaded view
|

Re: virtual transport lmtp vs. dovecot lda?

David Mehler
Hello,

I've put my configuration below. All my virtual users are under one
uid/gid 999, the vmail user. Should I do a db lookup or have the
uid/gid defined as static? Are my restrictions good? I'm running
Postfix 3.1 and the Dovecot version is 2.29. Are my message_size_limit
and mailbox_size_limit options right? Each virtual user has a database
256 MB quota. Any idea also on the mysql database connection issue?

Thanks.
Dave.

main.cf:
soft_bounce = no
queue_directory = /var/spool/postfix
command_directory = /usr/local/sbin
daemon_directory = /usr/local/libexec/postfix
data_directory = /var/db/postfix
mail_owner = postfix
myhostname = mail.example.com
mydomain = example.com
myorigin = $mydomain
inet_interfaces = xxx.xxx.xxx.xxx 127.0.0.1
mydestination = localhost
local_recipient_maps =
unknown_local_recipient_reject_code = 550
mynetworks = $config_directory/mynetworks
in_flow_delay = 1s
recipient_delimiter = +
smtpd_banner = $myhostname ESMTP
sendmail_path = /usr/local/sbin/sendmail
newaliases_path = /usr/local/bin/newaliases
mailq_path = /usr/local/bin/mailq
setgid_group = maildrop
html_directory = no
manpage_directory = /usr/local/man
sample_directory = /usr/local/etc/postfix
readme_directory = no

# Misc options
delay_warning_time = 4h
biff = no
bounce_template_file = /usr/local/etc/postfix/bounce.cf
smtp_helo_timeout = 60s
smtpd_soft_error_limit = 3
header_checks = pcre:/usr/local/etc/postfix/header_checks,
regexp:/usr/local/etc/postfix/phish419.regexp
mime_header_checks = regexp:/usr/local/etc/postfix/mime_header_checks
body_checks = regexp:/usr/local/etc/postfix/mbl-body-deny
hash_queue_depth = 2
hash_queue_names = incoming, hold defer deferred

# Virtual mailbox domains
virtual_mailbox_domains = proxy:mysql:/usr/local/etc/postfix/db/domains.cf
virtual_mailbox_maps = proxy:mysql:/usr/local/etc/postfix/db/mailboxes.cf
virtual_alias_maps = proxy:mysql:/usr/local/etc/postfix/db/aliases.cf
virtual_mailbox_base = /home/vmail
#virtual_uid_maps = static:999
#virtual_gid_maps = static:999
virtual_minimum_uid = 999
virtual_uid_maps = mysql:/usr/local/etc/postfix/db/uid.cf
virtual_gid_maps = mysql:/usr/local/etc/postfix/db/gid.cf
virtual_transport = lmtp:unix:private/dovecot-lmtp
#virtual_transport = dovecot
dovecot_destination_recipient_limit = 1

# For users who have moved
#relocated_maps = mysql:/usr/local/etc/postfix/db/relocated.cf

# Dovecot sasl authentication
smtpd_sasl_auth_enable = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_local_domain = $mydomain
broken_sasl_auth_clients = no
smtpd_sasl_security_options = noanonymous, noactive, nodictionary
smtpd_sasl_tls_security_options = noanonymous
# Shows to everyone the sasl authenticated username
smtpd_sasl_authenticated_header = yes

# uce
strict_rfc821_envelopes = yes
smtpd_helo_required = yes
disable_vrfy_command = yes
smtpd_reject_unlisted_sender = yes
show_user_unknown_table_name = no
unknown_address_reject_code  = 554
unknown_hostname_reject_code = 554
unknown_client_reject_code   = 554

smtpd_recipient_restrictions =
  permit_mynetworks
 permit_sasl_authenticated
  reject_unauth_destination
        check_helo_access hash:/usr/local/etc/postfix/helo_access,
        ,check_helo_access pcre:/usr/local/etc/postfix/helo_checks
        ,check_sender_mx_access cidr:/usr/local/etc/postfix/bogus_mx
 check_sender_access hash:/usr/local/etc/postfix/safe_addresses
 check_sender_access hash:/usr/local/etc/postfix/auto-whtlst
 check_client_access cidr:/usr/local/etc/postfix/spamfarms
 check_client_access cidr:/usr/local/etc/postfix/sinokorea.cidr
     permit_dnswl_client list.dnswl.org=127.0.[2..14].[1..3]
        check_reverse_client_hostname_access pcre:/usr/local/etc/postfix/fqrdns.pcre
 reject_unknown_reverse_client_hostname
  reject_non_fqdn_sender
 reject_non_fqdn_helo_hostname
 reject_invalid_helo_hostname
 reject_unknown_helo_hostname
 reject_unlisted_recipient
 reject_rhsbl_client dbl.spamhaus.org
 reject_rhsbl_sender dbl.spamhaus.org
 reject_rhsbl_helo dbl.spamhaus.org
  check_policy_service unix:private/spf-policy
# Postfix Quota status service
 #check_policy_service inet:127.0.0.1:12345
 check_policy_service unix:private/dovecot-quota

smtpd_data_restrictions = reject_unauth_pipelining

# TLS parameters
smtp_use_tls = yes
smtpd_use_tls = yes
smtpd_tls_auth_only = no
smtpd_tls_cert_file =
/usr/local/etc/letsencrypt/live/mail.example.com/fullchain.pem
smtpd_tls_key_file =
/usr/local/etc/letsencrypt/live/mail.example.com/privkey.pem
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtpd_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtpd_tls_mandatory_ciphers = high
smtpd_tls_exclude_ciphers = aNULL, eNULL, EXPORT, DES, RC4, MD5, PSK,
aECDH, EDH-DSS-DES-CBC3-SHA, EDH-RSA-DES-CDB3-SHA, KRB5-DES, CBC3-SHA
smtpd_tls_eecdh_grade = strong
# Offer opportunistic TLS (STARTTLS) to connections to this mail server.
smtpd_tls_security_level = may
# for smtpd pfs
smtpd_tls_dh1024_param_file = /etc/ssl/dhparam.pem
# I wanted a little more logging than default for incoming mail.
# Needed for tls verification
smtpd_tls_loglevel = 1
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
# Add TLS information to the message headers
smtpd_tls_received_header = yes
tls_preempt_cipherlist = yes
tls_high_cipherlist =
ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256@STRENGTH
# Use opportunistic TLS (STARTTLS) for outgoing mail
# if the remote server supports it.
smtp_tls_security_level = may
smtp_tls_protocols=!SSLv2,!SSLv3, !TLSv1
smtp_tls_mandatory_protocols=!SSLv2,!SSLv3, !TLSv1
smtp_tls_mandatory_ciphers = medium
smtp_tls_mandatory_exclude_ciphers = aNULL, eNULL, EXPORT, DES, RC4,
MD5, PSK, aECDH, EDH-DSS-DES-CBC3-SHA, EDH-RSA-DES-CDC3-SHA, KRB5-DE5,
CBC3-SHA
smtp_tls_cert_file = $smtpd_tls_cert_file
smtp_tls_key_file = $smtpd_tls_key_file
# I wanted a little more logging than default for outgoing mail.
smtp_tls_loglevel = 1
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

# For SPF
spf-policy_time_limit = 3600s

# OpenDKIM uses 8891
# OpenDMARC uses 8893
# pwhois-milter uses 8472
smtpd_milters = inet:127.0.0.1:8891,inet:127.0.0.1:8893,inet:127.0.0.1:8472
non_smtpd_milters = $smtpd_milters
milter_protocol = 6
milter_default_action = accept

# postscreen(8) settings
### Before-220 tests
postscreen_access_list = permit_mynetworks,
cidr:/usr/local/etc/postfix/postscreen_access.cidr,
cidr:/usr/local/etc/postfix/postscreen_spf_whitelist.cidr
postscreen_blacklist_action = drop
postscreen_dnsbl_action = enforce
postscreen_dnsbl_reply_map =
pcre:/usr/local/etc/postfix/postscreen_dnsbl_reply_map.pcre
postscreen_dnsbl_sites = zen.spamhaus.org*3
 b.barracudacentral.org*2
 bl.spameatingmonkey.net*2
   bl.spamcop.net
 dnsbl.sorbs.net
 psbl.surriel.com
 bl.mailspike.net
 swl.spamhaus.org*-4
 list.dnswl.org=127.[0..255].[0..255].0*-2
        list.dnswl.org=127.[0..255].[0..255].1*-3
        list.dnswl.org=127.[0..255].[0..255].[2..255]*-4
postscreen_dnsbl_threshold = 3
postscreen_greet_action = enforce
postscreen_dnsbl_whitelist_threshold = -1
### End of before-220 tests
### After-220 tests
### WARNING -- See "Tests after the 220 SMTP server greeting" in the
### Postscreen Howto and *UNDERSTAND* it *BEFORE* you enable the
### following tests!
#postscreen_bare_newline_action = drop
#postscreen_bare_newline_enable = yes
#postscreen_non_smtp_command_action = drop
#postscreen_non_smtp_command_enable = yes
#postscreen_pipelining_enable = yes
#postscreen_pipelining_action = drop
### ADDENDUM: Any one of the foregoing three *_enable settings may cause
### significant and annoying mail delays.
# For sharing a tempoary whitelist of addresses
postscreen_cache_map = proxy:btree:${data_directory}/postscreen_cache
postscreen_cache_cleanup_interval = 0

inet_protocols = ipv4
smtputf8_enable = no
compatibility_level = 9999
autoresponder_destination_recipient_limit = 1
meta_directory = /usr/local/libexec/postfix
shlib_directory = /usr/local/lib/postfix
message_size_limit = 52428800
mailbox_size_limit = 52428800
tls_ssl_options = NO_COMPRESSION

# Disable the rewriting of "site!user" into "user@site".
#swap_bangpath = no

# Disable the rewriting of the form "user%domain" to "user@domain".
#allow_percent_hack = no

# Allow recipient address start with '-'.
#allow_min_user = no

master.cf:
#smtp      inet  n       -       n       -       -       smtpd
smtp      inet  n       -       n       -       1       postscreen
smtpd     pass  -       -       n       -       -       smtpd
   -o content_filter=autoresponder:dummy
   -o content_filter=spamassassin
dnsblog   unix  -       -       n       -       0       dnsblog
tlsproxy  unix  -       -       n       -       0       tlsproxy
# Submission port 587 for client connection / sending mails from
authenticated users
submission inet n       -       n       -       -       smtpd
  -o syslog_name=postfix/submission
 # for opportunistic smtpd
  #-o smtpd_tls_security_level=may
 # Encrypt by default
  -o smtpd_tls_dh1024_param_file=/etc/ssl/dhparam.pem
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_sasl_type=dovecot
  -o smtpd_sasl_path=private/auth
  -o smtpd_reject_unlisted_recipient=no
  -o smtpd_recipient_restrictions=permit_sasl_authenticated,permit_mynetworks,reject
  -o tls_preempt_cipherlist=yes
#smtps     inet  n       -       n       -       -       smtpd
#  -o syslog_name=postfix/smtps
#  -o smtpd_tls_wrappermode=yes
#  -o smtpd_sasl_auth_enable=yes
#  -o smtpd_reject_unlisted_recipient=no
#  -o smtpd_client_restrictions=$mua_client_restrictions
#  -o smtpd_helo_restrictions=$mua_helo_restrictions
#  -o smtpd_sender_restrictions=$mua_sender_restrictions
#  -o smtpd_recipient_restrictions=
#  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
#  -o milter_macro_daemon_name=ORIGINATING
#628       inet  n       -       n       -       -       qmqpd
pickup    unix  n       -       n       60      1       pickup
cleanup   unix  n       -       n       -       0       cleanup
qmgr      unix  n       -       n       300     1       qmgr
#qmgr     unix  n       -       n       300     1       oqmgr
tlsmgr    unix  -       -       n       1000?   1       tlsmgr
rewrite   unix  -       -       n       -       -       trivial-rewrite
bounce    unix  -       -       n       -       0       bounce
defer     unix  -       -       n       -       0       bounce
trace     unix  -       -       n       -       0       bounce
verify    unix  -       -       n       -       1       verify
flush     unix  n       -       n       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
smtp      unix  -       -       n       -       -       smtp
relay     unix  -       -       n       -       -       smtp
#       -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
showq     unix  n       -       n       -       -       showq
error     unix  -       -       n       -       -       error
retry     unix  -       -       n       -       -       error
discard   unix  -       -       n       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       n       -       -       lmtp
anvil     unix  -       -       n       -       1       anvil
scache    unix  -       -       n       -       1       scache

# Dovecot local delivery agent - allows us to use sieve filters for spam
dovecot   unix  -       n       n       -       -       pipe
  flags=DRhu user=vmail:vmail argv=/usr/local/libexec/dovecot/deliver
-f ${sender} -d ${recipient}

# for SPF support
spf-policy unix -       n       n       -       0       spawn
          user=vmail argv=/usr/local/bin/perl
/usr/local/libexec/postfix-policyd-spf-perl

autoresponder unix - n n - - pipe
     flags=Fq user=autoresponse argv=/usr/local/sbin/autoresponse -s
${sender} -r ${recipient} -S ${sasl_username} -C ${client_address}

spamassassin unix - n n - - pipe
 flags=R user=spamd argv=/usr/local/bin/spamc -e /usr/sbin/sendmail
-oi -f ${sender} ${recipient}

dfilt     unix    -       n       n       -       -       pipe
    flags=Rq user=filter argv=/usr/local/etc/postfix/disclaimer -f
${sender} -r ${recipient}

# scan service for clamsmtpd
scan unix -       -       n       -       16       smtp
   -o smtp_data_done_timeout=1200
   -o smtp_send_xforward_command=yes
   -o disable_dns_lookups=yes

127.0.0.1:10026 inet n       -       n       -       16       smtpd
   -o content_filter=
   -o local_recipient_maps=
   -o relay_recipient_maps=
   -o smtpd_restriction_classes=
   -o smtpd_client_restrictions=
   -o smtpd_helo_restrictions=
   -o smtpd_sender_restrictions=
   -o smtpd_recipient_restrictions=permit_mynetworks,reject
   -o mynetworks_style=host
   -o smtpd_authorized_xforward_hosts=127.0.0.0/8



On 5/3/17, Noel Jones <[hidden email]> wrote:

> On 5/2/2017 6:33 PM, David Mehler wrote:
>> Hi,
>>
>> I'm not sure what to send. I've temporarily solved the problem by
>> increasing the mysql max_connections setting from 256 to 300 and
>> started working. Something is using up mysql processes when the lmtp
>> socket is used.
>>
>> Dave.
>
> Postfix makes lots of mysql connections. This is normal and expected.
>
> As Viktor already suggested, the solution is to use the postfix
> proxymap service to consolidate the mysql connections. This will
> greatly reduce the number of connections postfix makes to mysql and
> usually improve performance too.
>
> Documentation for proxymap is here:
> http://www.postfix.org/proxymap.8.html
> http://www.postfix.org/postconf.5.html#proxy_read_maps
>
> Using the proxymap service is really easy. Generally, everywhere in
> main.cf you have mysql:...  you replace with proxy:mysql:...
> In some cases you may need to alter the default value of
> proxy_read_maps.  There will be warnings in the log to guide you if
> this is needed.
>
>
>
>   -- Noel Jones
>
> ---
> This email has been checked for viruses by Avast antivirus software.
> https://www.avast.com/antivirus
>
>