Problems with rspamd, DKIM and a body getting altered after dkim signing because of changed content-transfer-encoding

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Problems with rspamd, DKIM and a body getting altered after dkim signing because of changed content-transfer-encoding

Michael Ludwig
Hi to all of you,

since weeks I'm struggling with this problem, not being able to solve
it on my own and I think the last possibility of getting help is to
ask you, the experts right here.
I set up a mailserver with the help of a howto I found on the net.
Mainly everything is okay, mails are received and sending is also
possible. Spam is getting sorted out and DKIM, SPF and DMARC is
working on the domains I switched to that new mailserver.

A few weeks ago I figured out that there is a problem when a
conversation is going on via mail, at some point the postfix changes
the content-transfer-encoding from 7bit to quoted-printable.
This makes sense, the MTA postfix is doing what it is expected to.
I've read about line-lengths, 8bitmime, utf8 and so on. But this
behaviour is breaking DKIM signing, as signing with rspamd is done
before the encoding conversion.

So I hope there is a possibility to get this problem solved with your help.
Please forgive me that at this point I did not post any details as I
am waiting for you telling me what configs, logs, snippets, headers
and so on you need to have in order to be able to help.
What I know is that rspamd is called via the milter functionality of
postfix. But I didn't find any ressources on the net on how to change
the order, so that rspamd dkim signing comes last.
This seems to be important in the master.cf as that must be the point
where the smtps-session is controlled. But I can't see anything where
it calls the milter, so that is why I have no clue where to begin
searching.

So any help is greatly appreciated.

Best regards,
Michael
Reply | Threaded
Open this post in threaded view
|

Re: Problems with rspamd, DKIM and a body getting altered after dkim signing because of changed content-transfer-encoding

Wietse Venema
Michael Ludwig:

> Hi to all of you,
>
> since weeks I'm struggling with this problem, not being able to solve
> it on my own and I think the last possibility of getting help is to
> ask you, the experts right here.
> I set up a mailserver with the help of a howto I found on the net.
> Mainly everything is okay, mails are received and sending is also
> possible. Spam is getting sorted out and DKIM, SPF and DMARC is
> working on the domains I switched to that new mailserver.
>
> A few weeks ago I figured out that there is a problem when a
> conversation is going on via mail, at some point the postfix changes
> the content-transfer-encoding from 7bit to quoted-printable.

No, it doesn't. Postfix may convert 8bit mail into 7bit quoted-printable,
depending on whether disable_mime_output_conversion is yes or no,
and whether a down-stream SMTP receiver anounces 8BITMIME support.

Postfix does not convert 7bit mail into quoted-printable. That is
how it has worked since 2002.

Please solve the right problem.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Problems with rspamd, DKIM and a body getting altered after dkim signing because of changed content-transfer-encoding

Ralph Seichter-2
In reply to this post by Michael Ludwig
* Michael Ludwig:

> So any help is greatly appreciated.

I doubt that Postfix is the culprit. I ran into similar issues a while
ago: E-Mail sent by me which included German umlauts did not arrive with
a valid DKIM signature. I thought that disabling the 8BITMIME extension
in Postfix was a possible solution, but that did not make a difference.

After fruitless experiments, I found that Thunderbird was (at least
partly) to blame. Only after setting "mail.strictly_mime=true" via TB's
config editor did I no longer experience DKIM signature breakage.

My point is that it is not trivial to figure out where your problem
originates. If you find a solution, I would be very interested hearing
about it.

-Ralph
Reply | Threaded
Open this post in threaded view
|

Re: Problems with rspamd, DKIM and a body getting altered after dkim signing because of changed content-transfer-encoding

Michael Ludwig
In reply to this post by Wietse Venema
Hello Wietse, and thank you for answering.

Indeed I experienced with the setting you mentioned,
disable_mime_output_conversion and set it to yes.
In fact this did not make any difference to the problem itself. Sorry
for not being precise enough on that 8bit / 7bit thing.

Postfix is converting the mail body to quoted-printable and I think it
also aligns the lines so that lines are not longer than X chars.
But it does so after the signing by rspamd is done. And that naturally
destroys the dkim signature for the body, resulting at the receiving
mail server marking the mail as junk because "message body has been
altered".

Please don't get me wrong, I absolutely think that Postfix is doing
nothing wrong here!
I just have the plan to change the order of things, so that rspamd
does the dkim signing after postfix changed the necessary things.Is
that possible when using milters for accessing rspamd?
Or is the order given by postfix and can't be changed as long as using
rspamd for signing?

Am Mi., 13. März 2019 um 01:07 Uhr schrieb Wietse Venema <[hidden email]>:
> Please solve the right problem.

Hope we can achieve exactly that. ;-)

Michael
Reply | Threaded
Open this post in threaded view
|

Re: Problems with rspamd, DKIM and a body getting altered after dkim signing because of changed content-transfer-encoding

Michael Ludwig
In reply to this post by Ralph Seichter-2
Hi Ralph,

thank you for your answer, too.

Am Mi., 13. März 2019 um 02:36 Uhr schrieb Ralph Seichter
<[hidden email]>:
> a valid DKIM signature. I thought that disabling the 8BITMIME extension
> in Postfix was a possible solution, but that did not make a difference.

That is what I tried, too. With the same outcome as the problem resided.

> After fruitless experiments, I found that Thunderbird was (at least
> partly) to blame. Only after setting "mail.strictly_mime=true" via TB's
> config editor did I no longer experience DKIM signature breakage.

That is a good point, but we all have to deal with buggy MUAs like TB
or Outlook.
I hate that too, but changing the default MUA config is not what I
intend to do, as postfix is able to convert the body part so that it
fits the RFCs.
I like that postfix can do such things and would love to stay with
those functions as postfix is rock solid with the most standard
settings.

> My point is that it is not trivial to figure out where your problem
> originates. If you find a solution, I would be very interested hearing

Fortunately it's the sequence of doing things as it seems to me.
Let's see if this order can be changed, that would do the trick.
Maybe Wietse or someone else is able to clarify and help. Would be
fantastic. ;-)

Michael
Reply | Threaded
Open this post in threaded view
|

Re: Problems with rspamd, DKIM and a body getting altered after dkim signing because of changed content-transfer-encoding

Dominic Raferd
On Wed, 13 Mar 2019 at 08:16, Michael Ludwig
<[hidden email]> wrote:
>
> Hi Ralph...

You seem to assume that postfix is the guilty party. Wietse wrote:
'Postfix does not convert 7bit mail into quoted-printable.' That is
definitive unless you produce evidence to the contrary. So what you
are experiencing must be caused by some other software. Maybe rspamd
itself, or another content filter.
Reply | Threaded
Open this post in threaded view
|

Re: Problems with rspamd, DKIM and a body getting altered after dkim signing because of changed content-transfer-encoding

Wietse Venema
In reply to this post by Michael Ludwig
Are you aware that the SMTP standard does not support lines > 1000
characters? If you send non-compliant email into Postfix or any other
mail server then you can expect DKIN signatures to break.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Problems with rspamd, DKIM and a body getting altered after dkim signing because of changed content-transfer-encoding

Bill Cole-3
In reply to this post by Michael Ludwig
On 12 Mar 2019, at 19:53, Michael Ludwig wrote:

> So any help is greatly appreciated.

You might get more specific and useful responses by following the
recommendations at http://www.postfix.org/DEBUG_README.html#mail

There are enough different ways that you MIGHT have Postfix configured
that without actual details of the configuration and logs showing what
Postfix is actually doing, it is a waste of time to try to guess at the
possibilities.


--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Reply | Threaded
Open this post in threaded view
|

Re: Problems with rspamd, DKIM and a body getting altered after dkim signing because of changed content-transfer-encoding

Michael Ludwig
Hello again.
Again my question: Is it possible to influence / to change the order?
So that postfix first does, what it has to do and then passes the
content to rspamd for dkim signing?

This is the output of postconf -n:

=============================================
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
append_dot_mydomain = no
biff = no
compatibility_level = 2
delay_warning_time = 4h
error_notice_recipient = [hidden email]
inet_interfaces = 1.2.3.4
inet_protocols = all
mailbox_command = procmail -a "$EXTENSION"
mailbox_size_limit = 0
message_size_limit = 20480000
milter_default_action = quarantine
milter_mail_macros = i {mail_addr} {client_addr} {client_name} {auth_authen}
milter_protocol = 6
mydestination = $myhostname, localhost.localdomain, localhost
myhostname = mymailserver.mydomain.com
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 1.2.3.4/28
myorigin = /etc/mailname
non_smtpd_milters = inet:127.0.0.1:11332
notify_classes = policy, resource, software, protocol
readme_directory = no
recipient_delimiter = +
relayhost =
smtp_tls_security_level = may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_banner = SMTP server at $myhostname
smtpd_client_restrictions = permit_sasl_authenticated,
permit_mynetworks, reject_unauth_pipelining
smtpd_data_restrictions = reject_unauth_pipelining
smtpd_helo_required = yes
smtpd_helo_restrictions = permit_sasl_authenticated,
permit_mynetworks, reject_non_fqdn_hostname,
reject_unknown_helo_hostname, reject_invalid_hostname
smtpd_milters = inet:127.0.0.1:11332
smtpd_recipient_restrictions = permit_sasl_authenticated,
permit_mynetworks, reject_unknown_recipient_domain,
check_policy_service inet:mymailserver.mydomain.com:22466
smtpd_relay_restrictions = permit_sasl_authenticated,
permit_mynetworks, reject_unauth_destination, check_policy_service
inet:mymailserver.mydomain.com:22466
smtpd_sasl_auth_enable = yes
smtpd_sasl_authenticated_header = yes
smtpd_sasl_path = private/auth
smtpd_sasl_security_options = noanonymous, noplaintext
smtpd_sasl_tls_security_options = noanonymous
smtpd_sasl_type = dovecot
smtpd_sender_restrictions = reject_unknown_sender_domain
smtpd_tls_auth_only = yes
smtpd_tls_cert_file = /etc/letsencrypt/live/webmail.mydomain.com/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/webmail.mydomain.com/privkey.pem
smtpd_tls_security_level = may
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtpd_use_tls = yes
virtual_alias_maps =
proxy:mysql:/etc/postfix/mysql/mysql_virtual_alias_maps.cf,
proxy:mysql:/etc/postfix/mysql/mysql_virtual_alias_domain_maps.cf,
proxy:mysql:/etc/postfix/mysql/mysql_virtual_alias_domain_catchall_maps.cf
virtual_mailbox_base = /var/vmail
virtual_mailbox_domains =
proxy:mysql:/etc/postfix/mysql/mysql_virtual_domains_maps.cf
virtual_mailbox_maps =
proxy:mysql:/etc/postfix/mysql/mysql_virtual_mailbox_maps.cf,
proxy:mysql:/etc/postfix/mysql/mysql_virtual_alias_domain_mailbox_maps.cf
virtual_transport = lmtp:unix:private/dovecot-lmtp
=============================================

Michael
Reply | Threaded
Open this post in threaded view
|

Re: Problems with rspamd, DKIM and a body getting altered after dkim signing because of changed content-transfer-encoding

Wietse Venema
Michael Ludwig:
> Hello again.
> Again my question: Is it possible to influence / to change the order?
> So that postfix first does, what it has to do and then passes the
> content to rspamd for dkim signing?

Maybe you can be more specific about what you want to happen before
Postfix hands off the email to rspamd, instead of after Postfix
receives the email from rspamd. If you are sending out-of-spec
email, then there are no guarantees. Not by Postfix and not by any
down-stream MTAs.

        Wietse