discarding EHLO keywords: CHUNKING

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

discarding EHLO keywords: CHUNKING

LoneStarKen
After updating from postfix.x86_64 2:3.3.1-12.el to postfix.x86_64 2:3.5.8-1.el8 I'm getting frequent log entries

Mar 19 10:51:58 mail postfix/smtpd[XXXXXX]: discarding EHLO keywords: CHUNKING

I understand an option is to disable BDAT, but I'd rather have BDAT working.  I'm not quite sure what the log entry indicates.  Do I need to install "CHUNKING" support somehow.  Here is my postconf -n output:

# postconf -n
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
broken_sasl_auth_clients = yes
command_directory = /usr/sbin
compatibility_level = 2
daemon_directory = /usr/libexec/postfix
data_directory = /var/lib/postfix
debug_peer_level = 2
debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin ddd $daemon_directory/$process_name $process_id & sleep 5
disable_vrfy_command = yes
dovecot_destination_recipient_limit = 1
html_directory = no
inet_interfaces = 96.126.122.158
inet_protocols = ipv4
mail_owner = postfix
mailbox_size_limit = 512000000
mailq_path = /usr/bin/mailq.postfix
manpage_directory = /usr/share/man
meta_directory = /etc/postfix
milter_default_action = accept
milter_protocol = 6
mydomain = lnstar.com
myhostname = mail.lnstar.com
mynetworks = 127.0.0.0/8 45.79.20.229/32 45.79.15.23/32 104.237.142.65/32 173.255.198.91/32 45.33.0.215/32 66.228.53.71/32 173.255.195.30/32 72.14.184.198/32 104.237.129.140/32 172.104.192.98/32
newaliases_path = /usr/bin/newaliases.postfix
non_smtpd_milters = $smtpd_milters
policyd-spf_time_limit = 3600
queue_directory = /var/spool/postfix
readme_directory = /usr/share/doc/postfix/README_FILES
sample_directory = /usr/share/doc/postfix/samples
sendmail_path = /usr/sbin/sendmail.postfix
setgid_group = postdrop
shlib_directory = /usr/lib64/postfix
smtp_tls_CAfile = /etc/pki/tls/certs/ca-bundle.crt
smtp_tls_CApath = /etc/pki/tls/certs
smtp_tls_cert_file = /etc/postfix/2019.pem
smtp_tls_key_file = /etc/postfix/2019.key
smtp_tls_loglevel = 1
smtp_tls_mandatory_protocols = !SSLv2,!SSLv3
smtp_tls_protocols = !SSLv2,!SSLv3
smtp_tls_security_level = may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_use_tls = yes
smtpd_client_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unknown_client_hostname, reject_rbl_client cbl.abuseat.org, reject_rbl_client b.barracudacentral.org
smtpd_data_restrictions = reject_unauth_pipelining
smtpd_delay_reject = yes
smtpd_helo_required = yes
smtpd_helo_restrictions = permit_sasl_authenticated, reject_non_fqdn_helo_hostname, reject_invalid_helo_hostname
smtpd_milters = inet:127.0.0.1:8891,inet:127.0.0.1:8893,unix:/run/spamass-milter/spamass-milter.sock
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, reject_unknown_recipient_domain, check_sender_access hash:/etc/postfix/access, reject_rbl_client zen.spamhaus.org, reject_rhsbl_reverse_client zen.spamhaus.org, reject_rhsbl_sender dbl.spamhaus.org, check_policy_service unix:private/policyd-spf, permit
smtpd_sasl_auth_enable = yes
smtpd_sasl_path = private/auth
smtpd_sasl_security_options = noanonymous
smtpd_sasl_type = dovecot
smtpd_tls_auth_only = yes
smtpd_tls_cert_file = /etc/postfix/2019.pem
smtpd_tls_key_file = /etc/postfix/2019.key
smtpd_tls_loglevel = 1
smtpd_tls_mandatory_protocols = !SSLv2,!SSLv3
smtpd_tls_protocols = !SSLv2,!SSLv3
smtpd_tls_received_header = yes
smtpd_tls_security_level = may
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtpd_tls_session_cache_timeout = 3600s
smtpd_use_tls = yes
tls_random_source = dev:/dev/urandom
unknown_local_recipient_reject_code = 550
virtual_alias_domains = [removed]
virtual_alias_maps = hash:/etc/postfix/virtusertable
virtual_transport = dovecot

http://www.postfix.org/BDAT_README.html isn't really helping determine the issue.

Thank you in advance for any clues you can provide!

Ken
Reply | Threaded
Open this post in threaded view
|

Re: discarding EHLO keywords: CHUNKING

Wietse Venema
LoneStarKen:
> After updating from postfix.x86_64 2:3.3.1-12.el to postfix.x86_64 2:3.5.8-1.el8 I'm getting frequent log entries
>
> Mar 19 10:51:58 mail postfix/smtpd[XXXXXX]: discarding EHLO keywords: CHUNKING

You have one or both of

    smtpd_discard_ehlo_keyword_address_maps
    smtpd_discard_ehlo_keywords

in main.cf or master.cf.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: discarding EHLO keywords: CHUNKING

Viktor Dukhovni
In reply to this post by LoneStarKen
On Fri, Mar 19, 2021 at 11:02:09AM -0500, LoneStarKen wrote:

> Mar 19 10:51:58 mail postfix/smtpd[XXXXXX]: discarding EHLO keywords: CHUNKING

Presumably you have a non-default setting of

    smtp_discard_ehlo_keywords

possibly via master.cf overrides?

--
    Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: discarding EHLO keywords: CHUNKING

LoneStarKen
In reply to this post by Wietse Venema
Hi Wietse,

Thank you for the response.  Those entries don't seem to exist in my main.cf or master.cf.

I grepped (case insensitive) main.cf for ehlo, keywords, and discard and none of those words exist.

The master.cf has a discard in the section "-o syslog_name=postfix/$service_name" though it isn't the phrase you mentioned.  I also note that the file timestamp is today (the date I ran the update). The contents of master.cf is below:

smtp      inet  n       -       n       -       -       smtpd
-o smtpd_sasl_auth_enable=no
submission inet n       -       n       -       -       smtpd
 -o syslog_name=postfix/submission
 -o smtpd_etrn_restrictions=reject
 -o smtpd_tls_security_level=encrypt
 -o smtpd_tls_auth_only=yes
 -o smtpd_client_restrictions=permit_mynetworks,permit_sasl_authenticated,reject
 -o smtpd_helo_restrictions=permit_mynetworks,permit
 -o milter_macro_daemon_name=ORIGINATING
 -o smtpd_sasl_type=dovecot
 -o smtpd_sasl_path=private/auth
smtps     inet  n       -       n       -       -       smtpd
 -o syslog_name=postfix/smtps
 -o smtpd_tls_wrappermode=yes
 -o smtpd_sasl_auth_enable=yes
 -o smtpd_relay_restrictions=permit_mynetworks,permit_sasl_authenticated,reject
 -o milter_macro_daemon_name=ORIGINATING
 -o smtpd_sasl_type=dovecot
 -o smtpd_sasl_path=private/auth
pickup    unix  n       -       n       60      1       pickup
cleanup   unix  n       -       n       -       0       cleanup
qmgr      unix  n       -       n       300     1       qmgr
tlsmgr    unix  -       -       n       1000?   1       tlsmgr
rewrite   unix  -       -       n       -       -       trivial-rewrite
bounce    unix  -       -       n       -       0       bounce
defer     unix  -       -       n       -       0       bounce
trace     unix  -       -       n       -       0       bounce
verify    unix  -       -       n       -       1       verify
flush     unix  n       -       n       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
smtp      unix  -       -       n       -       -       smtp
relay     unix  -       -       n       -       -       smtp
       -o syslog_name=postfix/$service_name
showq     unix  n       -       n       -       -       showq
error     unix  -       -       n       -       -       error
retry     unix  -       -       n       -       -       error
discard   unix  -       -       n       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       n       -       -       lmtp
anvil     unix  -       -       n       -       1       anvil
scache    unix  -       -       n       -       1       scache
dovecot   unix  -       n       n       -       -       pipe
   flags=DRhu user=vmail:vmail argv=/usr/libexec/dovecot/deliver -f ${sender} -d ${recipient}
policyd-spf  unix  -       n       n       -       0       spawn
   user=policyd-spf argv=/usr/libexec/postfix/policyd-spf
postlog   unix-dgram n  -       n       -       1       postlogd

Thanks again for your help!

Ken

> On Mar 19, 2021, at 1:01 PM, Wietse Venema <[hidden email]> wrote:
>
> LoneStarKen:
>> After updating from postfix.x86_64 2:3.3.1-12.el to postfix.x86_64 2:3.5.8-1.el8 I'm getting frequent log entries
>>
>> Mar 19 10:51:58 mail postfix/smtpd[XXXXXX]: discarding EHLO keywords: CHUNKING
>
> You have one or both of
>
>    smtpd_discard_ehlo_keyword_address_maps
>    smtpd_discard_ehlo_keywords
>
> in main.cf or master.cf.
>
> Wietse

Reply | Threaded
Open this post in threaded view
|

Re: discarding EHLO keywords: CHUNKING

Viktor Dukhovni
On Fri, Mar 19, 2021 at 01:48:53PM -0500, LoneStarKen wrote:

> Thank you for the response.  Those entries don't seem to exist in my main.cf or master.cf.
>
> I grepped (case insensitive) main.cf for ehlo, keywords, and discard and none of those words exist.

Are you running a modified Postfix with a non-empty default value of
$smtpd_discard_ehlo_keywords?  Check the output of "postconf" rather
than "postconf -n".

--
    Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: discarding EHLO keywords: CHUNKING

LoneStarKen
Hi Viktor,

Maybe so.  Here is output from postconf containing "discard_ehlo_keywords":

# postconf | grep discard_ehlo_keywords
postscreen_discard_ehlo_keywords = $smtpd_discard_ehlo_keywords
smtp_discard_ehlo_keywords =
smtpd_discard_ehlo_keywords = chunking

Looks like postscreen may somehow be the culprit.  Odd that the lines in main.cf and master.cf for postscreen are commented out. Maybe it's getting invoked some other way? (Full uncommented lines of master.cf in previous response.)
[master.cf snip]
  smtp      inet  n       -       n       -       -       smtpd
   -o smtpd_sasl_auth_enable=no
  #smtp      inet  n       -       n       -       1       postscreen
[/snip]

[main.cf snip]
#postscreen_greet_action = enforce
[/snip]

Recursive case insensitive grep from /etc/postfix for "keywords" returns nothing.  Not sure where I would change the setting (or if I should try to change the setting).

We are running postfix.x86_64 2:3.5.8-1.el8 on CentOS Stream release 8.

Output from postconf containing postscreen:
# postconf | grep postscreen
postscreen_access_list = permit_mynetworks
postscreen_bare_newline_action = ignore
postscreen_bare_newline_enable = no
postscreen_bare_newline_ttl = 30d
postscreen_blacklist_action = ignore
postscreen_cache_cleanup_interval = 12h
postscreen_cache_map = btree:$data_directory/postscreen_cache
postscreen_cache_retention_time = 7d
postscreen_client_connection_count_limit = $smtpd_client_connection_count_limit
postscreen_command_count_limit = 20
postscreen_command_filter =
postscreen_command_time_limit = ${stress?{10}:{300}}s
postscreen_disable_vrfy_command = $disable_vrfy_command
postscreen_discard_ehlo_keyword_address_maps = $smtpd_discard_ehlo_keyword_address_maps
postscreen_discard_ehlo_keywords = $smtpd_discard_ehlo_keywords
postscreen_dnsbl_action = ignore
postscreen_dnsbl_max_ttl = ${postscreen_dnsbl_ttl?{$postscreen_dnsbl_ttl}:{1}}h
postscreen_dnsbl_min_ttl = 60s
postscreen_dnsbl_reply_map =
postscreen_dnsbl_sites =
postscreen_dnsbl_threshold = 1
postscreen_dnsbl_timeout = 10s
postscreen_dnsbl_whitelist_threshold = 0
postscreen_enforce_tls = $smtpd_enforce_tls
postscreen_expansion_filter = $smtpd_expansion_filter
postscreen_forbidden_commands = $smtpd_forbidden_commands
postscreen_greet_action = ignore
postscreen_greet_banner = $smtpd_banner
postscreen_greet_ttl = 1d
postscreen_greet_wait = ${stress?{2}:{6}}s
postscreen_helo_required = $smtpd_helo_required
postscreen_non_smtp_command_action = drop
postscreen_non_smtp_command_enable = no
postscreen_non_smtp_command_ttl = 30d
postscreen_pipelining_action = enforce
postscreen_pipelining_enable = no
postscreen_pipelining_ttl = 30d
postscreen_post_queue_limit = $default_process_limit
postscreen_pre_queue_limit = $default_process_limit
postscreen_reject_footer = $smtpd_reject_footer
postscreen_reject_footer_maps = $smtpd_reject_footer_maps
postscreen_tls_security_level = $smtpd_tls_security_level
postscreen_upstream_proxy_protocol =
postscreen_upstream_proxy_timeout = 5s
postscreen_use_tls = $smtpd_use_tls
postscreen_watchdog_timeout = 10s
postscreen_whitelist_interfaces = static:all
proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps $virtual_alias_domains $virtual_mailbox_maps $virtual_mailbox_domains $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps $recipient_canonical_maps $relocated_maps $transport_maps $mynetworks $smtpd_sender_login_maps $sender_bcc_maps $recipient_bcc_maps $smtp_generic_maps $lmtp_generic_maps $alias_maps $smtpd_client_restrictions $smtpd_helo_restrictions $smtpd_sender_restrictions $smtpd_relay_restrictions $smtpd_recipient_restrictions $address_verify_sender_dependent_default_transport_maps $address_verify_sender_dependent_relayhost_maps $address_verify_transport_maps $fallback_transport_maps $lmtp_discard_lhlo_keyword_address_maps $lmtp_pix_workaround_maps $lmtp_sasl_password_maps $lmtp_tls_policy_maps $mailbox_command_maps $mailbox_transport_maps $postscreen_discard_ehlo_keyword_address_maps $rbl_reply_maps $sender_dependent_default_transport_maps $sender_dependent_relayhost_maps $smtp_discard_ehlo_keyword_address_maps $smtp_pix_workaround_maps $smtp_sasl_password_maps $smtp_tls_policy_maps $smtpd_discard_ehlo_keyword_address_maps $smtpd_milter_maps $virtual_gid_maps $virtual_uid_maps
proxy_write_maps = $smtp_sasl_auth_cache_name $lmtp_sasl_auth_cache_name $address_verify_map $postscreen_cache_map

Recommendations?  I don't understand how postscreen is configured or invoked and I'm still fuzzy on the ramifications of what the log entry is telling me.  Learning a bunch here!

Thanks for your help!
Ken

> On Mar 19, 2021, at 2:23 PM, Viktor Dukhovni <[hidden email]> wrote:
>
> On Fri, Mar 19, 2021 at 01:48:53PM -0500, LoneStarKen wrote:
>
>> Thank you for the response.  Those entries don't seem to exist in my main.cf or master.cf.
>>
>> I grepped (case insensitive) main.cf for ehlo, keywords, and discard and none of those words exist.
>
> Are you running a modified Postfix with a non-empty default value of
> $smtpd_discard_ehlo_keywords?  Check the output of "postconf" rather
> than "postconf -n".
>
> --
>    Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: discarding EHLO keywords: CHUNKING

Viktor Dukhovni
> On Mar 19, 2021, at 3:56 PM, LoneStarKen <[hidden email]> wrote:
>
> Maybe so.  Here is output from postconf containing "discard_ehlo_keywords":
>
> # postconf | grep discard_ehlo_keywords
> postscreen_discard_ehlo_keywords = $smtpd_discard_ehlo_keywords
> smtp_discard_ehlo_keywords =
> smtpd_discard_ehlo_keywords = chunking
>
> Looks like postscreen may somehow be the culprit.  Odd that the lines in main.cf
> and master.cf for postscreen are commented out. Maybe it's getting invoked some
> other way? (Full uncommented lines of master.cf in previous response.)

No postscreen(8) has nothing to do with it, it just defaults to whatever
smtpd(8) defaults to.  The real issue is that you have (whether as a
compiled in default, or in fact in main.cf) a non-empty setting for:

        smtpd_discard_ehlo_keywords = chunking

if that's also reported by "postconf -d", then whoever compiled your Postfix
package decided to change the upstream default.  If you don't like that choice
you can set an explicit empty value in main.cf and complain to your package
maintainers:

        smtpd_discard_ehlo_keywords =

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: discarding EHLO keywords: CHUNKING

Wietse Venema
In reply to this post by LoneStarKen
LoneStarKen:
> smtpd_discard_ehlo_keywords = chunking

Well there is your problem. If you did not configure this, i.e.
"postonf -d smtpd_discard_ehlo_keywords" shows "chunking", then
complain to your vendor.

Otherwise, how many (Postfix) master daemons are there on your system?

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: discarding EHLO keywords: CHUNKING

LoneStarKen
Hi Wietse,

I think I just have one postfix master:

# ps aux | grep postfix
root        1693  0.0  0.1 123108  6356 ?        Ss   13:31   0:00 /usr/libexec/postfix/master -w
postfix     1703  0.0  0.2 150548 11020 ?        S    13:31   0:00 qmgr -l -t unix -u
postfix     4340  0.0  0.3 154824 12300 ?        S    13:31   0:00 tlsmgr -l -t unix -u
postfix     4395  0.0  0.2 150412 10064 ?        S    13:31   0:00 anvil -l -t unix -u
postfix     7795  0.0  0.2 150412 10168 ?        S    15:11   0:00 pickup -l -t unix -u
postfix     7916  0.0  0.2 150424 10428 ?        S    15:18   0:00 trivial-rewrite -n rewrite -t unix -u
postfix     7968  0.0  0.2 123048  9056 ?        S    15:20   0:00 spawn -z -n policyd-spf -t unix user=policyd-spf argv=/usr/libexec/postfix/policyd-spf
postfix     8361  0.0  0.4 160468 15856 ?        S    15:39   0:00 smtpd -n smtp -t inet -u -o stress= -o smtpd_sasl_auth_enable=no
policyd+    8388  0.0  0.4  77440 15648 ?        Ss   15:40   0:00 /usr/bin/python3.6 -s /usr/libexec/postfix/policyd-spf
root        8436  0.0  0.0  12136  1124 pts/0    S+   15:43   0:00 grep --color=auto postfix

Thanks,
Ken

> On Mar 19, 2021, at 3:40 PM, Wietse Venema <[hidden email]> wrote:
>
> LoneStarKen:
>> smtpd_discard_ehlo_keywords = chunking
>
> Well there is your problem. If you did not configure this, i.e.
> "postonf -d smtpd_discard_ehlo_keywords" shows "chunking", then
> complain to your vendor.
>
> Otherwise, how many (Postfix) master daemons are there on your system?
>
> Wietse

Reply | Threaded
Open this post in threaded view
|

Re: discarding EHLO keywords: CHUNKING

LoneStarKen
In reply to this post by Viktor Dukhovni
Hi Viktor,

# postconf -d | grep smtpd_discard
postscreen_discard_ehlo_keyword_address_maps = $smtpd_discard_ehlo_keyword_address_maps
postscreen_discard_ehlo_keywords = $smtpd_discard_ehlo_keywords
proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps $virtual_alias_domains $virtual_mailbox_maps $virtual_mailbox_domains $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps $recipient_canonical_maps $relocated_maps $transport_maps $mynetworks $smtpd_sender_login_maps $sender_bcc_maps $recipient_bcc_maps $smtp_generic_maps $lmtp_generic_maps $alias_maps $smtpd_client_restrictions $smtpd_helo_restrictions $smtpd_sender_restrictions $smtpd_relay_restrictions $smtpd_recipient_restrictions $address_verify_sender_dependent_default_transport_maps $address_verify_sender_dependent_relayhost_maps $address_verify_transport_maps $fallback_transport_maps $lmtp_discard_lhlo_keyword_address_maps $lmtp_pix_workaround_maps $lmtp_sasl_password_maps $lmtp_tls_policy_maps $mailbox_command_maps $mailbox_transport_maps $postscreen_discard_ehlo_keyword_address_maps $rbl_reply_maps $sender_dependent_default_transport_maps $sender_dependent_relayhost_maps $smtp_discard_ehlo_keyword_address_maps $smtp_pix_workaround_maps $smtp_sasl_password_maps $smtp_tls_policy_maps $smtpd_discard_ehlo_keyword_address_maps $smtpd_milter_maps $virtual_gid_maps $virtual_uid_maps
smtpd_discard_ehlo_keyword_address_maps =
smtpd_discard_ehlo_keywords = chunking

# dnf info postfix
Last metadata expiration check: 1:43:17 ago on Fri 19 Mar 2021 02:06:32 PM CDT.
Installed Packages
Name         : postfix
Epoch        : 2
Version      : 3.5.8
Release      : 1.el8
Architecture : x86_64
Size         : 4.4 M
Source       : postfix-3.5.8-1.el8.src.rpm
Repository   : @System
From repo    : baseos
Summary      : Postfix Mail Transport Agent
URL          : http://www.postfix.org
License      : (IBM and GPLv2+) or (EPL-2.0 and GPLv2+)
Description  : Postfix is a Mail Transport Agent (MTA).

So I guess it was compiled in the Postfix package.

What are the ramifications of "smtpd_discard_ehlo_keywords = chunking" vs "smtpd_discard_ehlo_keywords = "?  Is the package manager disabling BDAT by discarding chunking?  Sorry, I'm not finding much info on BDAT.  I'm not sure if a module is needed that isn't installed to implement BDAT or there is some other reason the package manager might have disabled it.  Not sure if we need it or will need it in the future for some reason.

Thanks for all your expert help!
Ken

> On Mar 19, 2021, at 3:34 PM, Viktor Dukhovni <[hidden email]> wrote:
>
> No postscreen(8) has nothing to do with it, it just defaults to whatever
> smtpd(8) defaults to.  The real issue is that you have (whether as a
> compiled in default, or in fact in main.cf) a non-empty setting for:
>
> smtpd_discard_ehlo_keywords = chunking
>
> if that's also reported by "postconf -d", then whoever compiled your Postfix
> package decided to change the upstream default.  If you don't like that choice
> you can set an explicit empty value in main.cf and complain to your package
> maintainers:
>
> smtpd_discard_ehlo_keywords =
>
> --
> Viktor.
>

Reply | Threaded
Open this post in threaded view
|

Re: discarding EHLO keywords: CHUNKING

Viktor Dukhovni
On Fri, Mar 19, 2021 at 04:14:30PM -0500, LoneStarKen wrote:

> # postconf -d | grep smtpd_discard
> smtpd_discard_ehlo_keywords = chunking

That's the only value needed.  Whoever built your package decided to
disable the ESMTP CHUNKING extension (aka BDAT).  If you want/need
BDAT, you'll need to set that parameter explicitly empty.

> # dnf info postfix
> Version      : 3.5.8
> Source       : postfix-3.5.8-1.el8.src.rpm
> From repo    : baseos

Blame the RedHat/Fedora/CentOS Postfix maintainers.

> What are the ramifications of "smtpd_discard_ehlo_keywords = chunking"
> vs "smtpd_discard_ehlo_keywords = "?

The server does not advertise support for CHUNKING and therefore clients
cannot use the BDAT command (which is defined as part of the CHUNKING
ESMTP extension in RFC3030).

> I'm not sure if a module is needed that isn't installed to
> implement BDAT or there is some other reason the package manager might
> have disabled it.

There may have been some early bugs in the Exim client-side
implementation of BDAT that motivated someone to do this.  I am not
aware of any ongoing issues.

--
    Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: discarding EHLO keywords: CHUNKING

LoneStarKen
Thank you for all the help Viktor.  Based on your advice, I decided
the package maintainer probably had some reason to disable
CHUNKING so I just added the following to main.cf to quiet the
logging:

smtpd_discard_ehlo_keywords = chunking, silent-discard

I also entered a bug in bugs.centos.org requesting clarification on
the decision to disable CHUNKING.
Reply | Threaded
Open this post in threaded view
|

Re: discarding EHLO keywords: CHUNKING

Matus UHLAR - fantomas
On 20.03.21 08:38, LoneStarKen wrote:
>Thank you for all the help Viktor.  Based on your advice, I decided
>the package maintainer probably had some reason to disable
>CHUNKING so I just added the following to main.cf to quiet the
>logging:
>
>smtpd_discard_ehlo_keywords = chunking, silent-discard
>
>I also entered a bug in bugs.centos.org requesting clarification on
>the decision to disable CHUNKING.

wouldn't is be better to enable chunking by setting it to empty?

smtpd_discard_ehlo_keywords =

especially when you want to have BDAT working...
--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Microsoft dick is soft to do no harm
Reply | Threaded
Open this post in threaded view
|

Re: discarding EHLO keywords: CHUNKING

LoneStarKen
Possibly.  Since I am unsure why the package maintainer disabled
CHUNKING I am concerned enabling it, we might have a broken
implementation of BDAT or even worse something else breaks.
Since this is a production server, I'm going to err on the
side of caution until I get some clarification from the
package maintainer regarding the decision to disable it.

In addition, I have been unable to find enough information on
BDAT to feel comfortable I know how it should work and how
to test it in the event we decided to enable it.

> On Mar 20, 2021, at 9:01 AM, Matus UHLAR - fantomas <[hidden email]> wrote:
>
> On 20.03.21 08:38, LoneStarKen wrote:
>> Thank you for all the help Viktor.  Based on your advice, I decided
>> the package maintainer probably had some reason to disable
>> CHUNKING so I just added the following to main.cf to quiet the
>> logging:
>>
>> smtpd_discard_ehlo_keywords = chunking, silent-discard
>>
>> I also entered a bug in bugs.centos.org requesting clarification on
>> the decision to disable CHUNKING.
>
> wouldn't is be better to enable chunking by setting it to empty?
>
> smtpd_discard_ehlo_keywords =
> especially when you want to have BDAT working...
> --
> Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
> Warning: I wish NOT to receive e-mail advertising to this address.
> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
> Microsoft dick is soft to do no harm

Reply | Threaded
Open this post in threaded view
|

Re: discarding EHLO keywords: CHUNKING

Benny Pedersen-2
On 2021-03-20 15:22, LoneStarKen wrote:

> In addition, I have been unable to find enough information on
> BDAT to feel comfortable I know how it should work and how
> to test it in the event we decided to enable it.

thank you for using postfix, its stable code in the first place unless
you can show examples of not stable, one could drop postfix if exim or
sendmail is more stable, but i have used postfix last 30 years or so,
its stable for me at least

i am just shooked to see you trust centos more then maintainers of
postfix c code
Reply | Threaded
Open this post in threaded view
|

Re: discarding EHLO keywords: CHUNKING

John Levine
In reply to this post by LoneStarKen
It appears that LoneStarKen <[hidden email]> said:

>Possibly.  Since I am unsure why the package maintainer disabled
>CHUNKING I am concerned enabling it, we might have a broken
>implementation of BDAT or even worse something else breaks.
>Since this is a production server, I'm going to err on the
>side of caution until I get some clarification from the
>package maintainer regarding the decision to disable it.
>
>In addition, I have been unable to find enough information on
>BDAT to feel comfortable I know how it should work and how
>to test it in the event we decided to enable it.

It's defined in RFC 3030.  Read all about it: https://www.rfc-editor.org/info/rfc3030

It happens that I just added CHUNKING and BDAT to an MTA I use (mailfront if you know
what that is.)  Inbound the code is quite simple and I would be surprised if there were
any problems with it.  Outbound it's a little trickier since BDAT requires you to know
the exact size of the chunk of message you're sending, which means it has to deal with
turning \n into \r\n, but again, it's not a big deal.

I'm guessing that someone had some problem talking to Gmail or Hotmail/Outlook, the
two largest systems that can use BDAT, turned it off to see if that was the problem,
and never bothered to turn it back on when it wasn't.

To test it, turn it on, send yourself a fairly large message from a Gmail account,
and see if you get it.

R's,
John
Reply | Threaded
Open this post in threaded view
|

BINARYMIME in Postfix

Demi Marie Obenour
On 3/20/21 2:51 PM, John Levine wrote:

> It's defined in RFC 3030.  Read all about it: https://www.rfc-editor.org/info/rfc3030
>
> It happens that I just added CHUNKING and BDAT to an MTA I use (mailfront if you know
> what that is.)  Inbound the code is quite simple and I would be surprised if there were
> any problems with it.  Outbound it's a little trickier since BDAT requires you to know
> the exact size of the chunk of message you're sending, which means it has to deal with
> turning \n into \r\n, but again, it's not a big deal.
>
> I'm guessing that someone had some problem talking to Gmail or Hotmail/Outlook, the
> two largest systems that can use BDAT, turned it off to see if that was the problem,
> and never bothered to turn it back on when it wasn't.
This reminded me that Postfix does not have BINARYMIME support.
GMail does not, but Outlook does.

How useful would BINARYMIME support be?  It does mean that DKIM signing
would need to be done in the sending path, but I cannot think of any
reasons that would be a blocker.  Having DKIM and DMARC built-in to
Postfix would be a nice feature, tbh.  The only open-source MTA I
know of with built-in DKIM is Exim but I would never dare use it in
production.

Ideally, the signing keys should be in a separate process for privilege
separation, but Postfix is already multi-process so that should be
doable.  Of course, the final decision is up to Wietse.

Sincerely,

Demi Marie Obenour
she/her/hers

OpenPGP_0xB288B55FFF9C22C1.asc (5K) Download Attachment
OpenPGP_signature (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: BINARYMIME in Postfix

Wietse Venema
Demi Marie Obenour:

> How useful would BINARYMIME support be?  It does mean that DKIM signing
> would need to be done in the sending path, but I cannot think of any
> reasons that would be a blocker.  Having DKIM and DMARC built-in to
> Postfix would be a nice feature, tbh.  The only open-source MTA I
> know of with built-in DKIM is Exim but I would never dare use it in
> production.
>
> Ideally, the signing keys should be in a separate process for privilege
> separation, but Postfix is already multi-process so that should be
> doable.  Of course, the final decision is up to Wietse.

See also https://github.com/OpenSMTPD/OpenSMTPD/issues/1090

I haven't had the time to properly analyze what this would take.
Here's a first inventory.

- Background: there is no 1:1 relationship between BINARYMIME chunks
and MIME segments. One incoming BINARYMIME chunk can contain any
combination of MIME text segments and MIME binary segments, and one
large MIME segment can span multiple chunks.

- Background: Postfix does not store text line endings (<LF> or
<CR><LF>) internally. When receiving mail, the Postfix SMTP server
strips <CR><LF> line endings from text, and the Postfix sendmail
command strips <LF> line endings when ingesting mail submitted with
UNIX command-line tools. When delivering mail, Postfix appends <LF>
line endings when writing message text to a UNIX file or command,
and Postfix appends <CR><LF> line endings when writing message text
to a remote SMTP server. Line endings are not a message property;
instead, they are a property of the external environment, to be
added by Postfix delivery agents and to be stripped by Postfix mail
receiving programs.

- Impact: in a receiving Postfix server, the Postfix MIME parser
needs to be changed to strip <CR><LF> line endings from MIME headers
and from lines in MIME text segments, and the Postfix SMTP server
needs to be changed to stop stripping those <CR><LF> line endings.

- Impact: the Postfix MIME parser needs new support for 'type binary'
MIME body content, and it must parse such content not as lines of
text ending in <CR><LF><CR><LF>--boundary-string", but as a binary
blob with a length determined by a preceding Content-Length MIME
header.

- Impact: binary blobs need to be passed around internally in Postfix
as a new type of record, so that they remain distinct from Postfix
text records. Specifically, the smtpd daemon sends BINARYMIME chunks
as blobs to the cleanup daemon (and maybe also other message content).

- The impact on Milters or other extrnal content filters is unknown.

- The impact on smtpd_proxy_filter is unknown. Currently, the Postfix
SMTP daemon replaces a sequence of BDAT commands with a single DATA
command. That can't work with BINARYMIME, because BINARYMIME is
fundamentally incompatible with DATA.

- The impact on UNIX mailbox stores is unknown (these store multiple
messages per file). Every mail reading program will have to understand
MIME and will have to parse Content-Length: headers or else it will
be hopelessly confused about where a message ends.

That's just a few things off the top of my head.:1

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: BINARYMIME in Postfix

John Levine
It appears that Wietse Venema <[hidden email]> said:

>Demi Marie Obenour:
>> How useful would BINARYMIME support be?  It does mean that DKIM signing
>> would need to be done in the sending path, but I cannot think of any
>> reasons that would be a blocker.  Having DKIM and DMARC built-in to
>> Postfix would be a nice feature, tbh.  The only open-source MTA I
>> know of with built-in DKIM is Exim but I would never dare use it in
>> production.
>>
>> Ideally, the signing keys should be in a separate process for privilege
>> separation, but Postfix is already multi-process so that should be
>> doable.  Of course, the final decision is up to Wietse.
>
>See also https://github.com/OpenSMTPD/OpenSMTPD/issues/1090
>
>I haven't had the time to properly analyze what this would take.
>Here's a first inventory.

Since most other MTAs don't suport BINARYMIME, you'd need some way to
sniff the recipient server and then rewrite the binary parts as base64
before sending the message, recomputing any DKIM and ARC signatures on the way.
Have fun with that.

BINARYMIME avoids the 33% size increase of base64.  If people cared about that,
since every MTA now supports 8BITMIME it would be easy to invent a quoted-unprintable
content-transfer-encoding which escaped only the few characters that are special
in 8BITMIME (CR LF NUL and to be on the safe side, 0xff.)  That would get you about
98% of the way to binary with 2% of the work.

R's,
John
Reply | Threaded
Open this post in threaded view
|

Re: BINARYMIME in Postfix

Wietse Venema
John Levine:

> It appears that Wietse Venema <[hidden email]> said:
> >Demi Marie Obenour:
> >> How useful would BINARYMIME support be?  It does mean that DKIM signing
> >> would need to be done in the sending path, but I cannot think of any
> >> reasons that would be a blocker.  Having DKIM and DMARC built-in to
> >> Postfix would be a nice feature, tbh.  The only open-source MTA I
> >> know of with built-in DKIM is Exim but I would never dare use it in
> >> production.
> >>
> >> Ideally, the signing keys should be in a separate process for privilege
> >> separation, but Postfix is already multi-process so that should be
> >> doable.  Of course, the final decision is up to Wietse.
> >
> >See also https://github.com/OpenSMTPD/OpenSMTPD/issues/1090
> >
> >I haven't had the time to properly analyze what this would take.
> >Here's a first inventory.
>
> Since most other MTAs don't suport BINARYMIME, you'd need some way
> to sniff the recipient server and then rewrite the binary parts
> as base64 before sending the message, recomputing any DKIM and ARC
> signatures on the way.  Have fun with that.

That might work for one-to-one messages, but not for one-to-many.
A more general solution would convert and sign during delivery. Not
fun, but technically possible.

> BINARYMIME avoids the 33% size increase of base64.  If people cared
> about that, since every MTA now supports 8BITMIME it would be easy
> to invent a quoted-unprintable content-transfer-encoding which
> escaped only the few characters that are special in 8BITMIME (CR
> LF NUL and to be on the safe side, 0xff.)  That would get you about
> 98% of the way to binary with 2% of the work.

This would turn binary content into a long line. That works perfectly
with qmail and Postfix (except that the Postfix SMTP client will
need a hint to avoid folding such lines at the 998 octet limit of
RFC 5321).

        Wietse
12