Re: dkim + altermime disclaimer

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: dkim + altermime disclaimer

Mark Martinec-5
(crossposted from the amavis-user ML)

Patrick Wong writes:

> I have a situation where altermime disclaimer insertion and amavis dkim
> signing on outgoing mail begot result from gmail's dkim verification I
> don't know what to make of:
>
> Mail of mime type plaintext with plaintext disclaimer inserted through
> altermime + dkim signing is OK when verified by gmail (dkim=pass)
>
> But, when mime type is html and html disclaimer is used, I'll get
> "dkim=neutral (body hash did not verify)" on gmail's authentication
> result.

I was able to reproduce the problem and I understand what is happening,
although I'm not sure which component is to blame.

When altermime is inserting a plain text disclaimer to a mail text
(as stored on a file on Unix, i.e. line endings are LF), it inserts
a text from a disclaimer file as-is, i.e. a disclaimer text along with
its LF line endings is directly inserted into a mail text, unmodified.
This works fine, message transfers unmodified and a signature is valid.

When altermime is inserting a '--disclaimer-html' into a html mail,
for some reason it inserts a CR before each LF of the inserted html text,
instead of copying it as-is from a disclaimer file! Here is an example
of the resulting message body:

  Content-Type: text/html; charset="iso-8859-1"

  <html><head><meta name="qrichtext" content="1" /></head><body
   style="font-size:18pt;font-family:Bitstream Vera Sans Mono">
  <p>testing</p>
  <p><span style="font-style:italic">ital</span></p>
  ^M
  <br>^M
  <i>Disclaimer</i>^M
  <p>one, two, three^M
  <br>^M
  </body></html>
  --Boundary-01=_cBVUIiha08fkdho--

Amavisd then signs the resulting message, along with all the extra CR
in the disclaimer part of a html text, then sends it over SMTP to Postfix.
I confirmed that a signature is correct for the presented message,
and tcpdump confirms that the SMTP session still has a CR before each
end-of-line. As the SMTP protocol demands line endings to be CRLF,
the bytes sent are actually CR CRLF (0d 0d 0a):

62 72 3e 0d 0d 0a 3c 2f 62 6f 64 79 3e 3c 2f 68    br>...</body></h

When Postfix relays such a message, it strips off the extra CRs
(CR CR LF => CR LF) as confirmed by a tcpdump of a SMTP session
towards the next MTA. This modification of the message breaks
a DKIM signature.


Now, an easy finger-pointing tells me that altermime shouldn't
be inserting extra CRs in the HTML disclaimer part (like it
does correctly for the plain text disaclaimer).

Apart from the altermime's guilt, which is the next in line?
How should MTA and a DKIM-signer behave regarding bare-CR in
a mail body?

  Mark
Reply | Threaded
Open this post in threaded view
|

Re: dkim + altermime disclaimer

Noel Jones-2
Mark Martinec wrote:
>
> When Postfix relays such a message, it strips off the extra CRs
> (CR CR LF => CR LF) as confirmed by a tcpdump of a SMTP session
> towards the next MTA. This modification of the message breaks
> a DKIM signature.
>

Please verify and report that:
message_strip_characters =
is empty (which is the default) in your postconf output and
not used in master.cf.

Other than that, I got nothin.

--
Noel Jones
Reply | Threaded
Open this post in threaded view
|

Re: dkim + altermime disclaimer

mouss-2
In reply to this post by Mark Martinec-5
Mark Martinec wrote:

> (crossposted from the amavis-user ML)
>
> Patrick Wong writes:
>  
>> I have a situation where altermime disclaimer insertion and amavis dkim
>> signing on outgoing mail begot result from gmail's dkim verification I
>> don't know what to make of:
>>
>> Mail of mime type plaintext with plaintext disclaimer inserted through
>> altermime + dkim signing is OK when verified by gmail (dkim=pass)
>>
>> But, when mime type is html and html disclaimer is used, I'll get
>> "dkim=neutral (body hash did not verify)" on gmail's authentication
>> result.
>>    
>
> I was able to reproduce the problem and I understand what is happening,
> although I'm not sure which component is to blame.
>
> When altermime is inserting a plain text disclaimer to a mail text
> (as stored on a file on Unix, i.e. line endings are LF), it inserts
> a text from a disclaimer file as-is, i.e. a disclaimer text along with
> its LF line endings is directly inserted into a mail text, unmodified.
> This works fine, message transfers unmodified and a signature is valid.
>
> When altermime is inserting a '--disclaimer-html' into a html mail,
> for some reason it inserts a CR before each LF of the inserted html text,
>  

so it sends lines ending in CRLF? if so, this looks ok to me. or am I
misunderstanding?

> instead of copying it as-is from a disclaimer file! Here is an example
> of the resulting message body:
>
>   Content-Type: text/html; charset="iso-8859-1"
>
>   <html><head><meta name="qrichtext" content="1" /></head><body
>    style="font-size:18pt;font-family:Bitstream Vera Sans Mono">
>   <p>testing</p>
>   <p><span style="font-style:italic">ital</span></p>
>   ^M
>   <br>^M
>   <i>Disclaimer</i>^M
>   <p>one, two, three^M
>   <br>^M
>   </body></html>
>   --Boundary-01=_cBVUIiha08fkdho--
>
> Amavisd then signs the resulting message, along with all the extra CR
> in the disclaimer part of a html text, then sends it over SMTP to Postfix.
> I confirmed that a signature is correct for the presented message,
> and tcpdump confirms that the SMTP session still has a CR before each
> end-of-line. As the SMTP protocol demands line endings to be CRLF,
> the bytes sent are actually CR CRLF (0d 0d 0a):
>  

the question is why this becomes a CRCRLF when it was simply a CRLF?

> 62 72 3e 0d 0d 0a 3c 2f 62 6f 64 79 3e 3c 2f 68    br>...</body></h
>
> When Postfix relays such a message, it strips off the extra CRs
> (CR CR LF => CR LF) as confirmed by a tcpdump of a SMTP session
> towards the next MTA. This modification of the message breaks
> a DKIM signature.
>
>
> Now, an easy finger-pointing tells me that altermime shouldn't
> be inserting extra CRs in the HTML disclaimer part (like it
> does correctly for the plain text disaclaimer).
>  

That's debatable. if the lines it sends end in a simple CRLF, then input
is ok. even if running under unix, a program can check its input before
adding a CR.
> Apart from the altermime's guilt, which is the next in line?
> How should MTA and a DKIM-signer behave regarding bare-CR in
> a mail body?
>
>   Mark
>  

Reply | Threaded
Open this post in threaded view
|

Re: dkim + altermime disclaimer

Wietse Venema
In reply to this post by Mark Martinec-5
Mark Martinec:
> Now, an easy finger-pointing tells me that altermime shouldn't
> be inserting extra CRs in the HTML disclaimer part (like it
> does correctly for the plain text disaclaimer).
>
> Apart from the altermime's guilt, which is the next in line?
> How should MTA and a DKIM-signer behave regarding bare-CR in
> a mail body?

Bare CR is forbidden by RFC, so any attempt to "validate"
is doomed to fail.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: dkim + altermime disclaimer

Wietse Venema
In reply to this post by Noel Jones-2
Noel Jones:

> Mark Martinec wrote:
> >
> > When Postfix relays such a message, it strips off the extra CRs
> > (CR CR LF => CR LF) as confirmed by a tcpdump of a SMTP session
> > towards the next MTA. This modification of the message breaks
> > a DKIM signature.
> >
>
> Please verify and report that:
> message_strip_characters =
> is empty (which is the default) in your postconf output and
> not used in master.cf.
>
> Other than that, I got nothin.

Postfix SMTPD strips extraneous CR before LF, because a) some
Windows mail software could not handle it and b) it is invalid
SMTP.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: dkim + altermime disclaimer

Mark Martinec-5
In reply to this post by mouss-2
mouss,

> > When altermime is inserting a '--disclaimer-html' into a html mail,
> > for some reason it inserts a CR before each LF of the inserted html text,
>
> so it sends lines ending in CRLF? if so, this looks ok to me. or am I
> misunderstanding?

Just to clarify: altermime is dealing with plain text files
(it doesn't transfer anything by itself), i.e. with native
Unix line-endings text files. An original mail and a disclaimer
text reside on their own native text files, and the resulting mail
ends up as a native Unix text file. There is no place for foreign
line terminators to creep-in into the equation in one section
of the resulting file.

> the question is why this becomes a CRCRLF when it was simply a CRLF?

SMTP mandates that line endings should look on the wire as a CRLF pair.
So a native end-of-line (NL) is translated into CRLF, and a standalone
extra CR is kept unmodified.

  Mark
Reply | Threaded
Open this post in threaded view
|

Re: dkim + altermime disclaimer

Victor Duchovni
On Fri, Jun 13, 2008 at 12:45:54AM +0200, Mark Martinec wrote:

> mouss,
>
> > > When altermime is inserting a '--disclaimer-html' into a html mail,
> > > for some reason it inserts a CR before each LF of the inserted html text,
> >
> > so it sends lines ending in CRLF? if so, this looks ok to me. or am I
> > misunderstanding?
>
> Just to clarify: altermime is dealing with plain text files
> (it doesn't transfer anything by itself), i.e. with native
> Unix line-endings text files. An original mail and a disclaimer
> text reside on their own native text files, and the resulting mail
> ends up as a native Unix text file. There is no place for foreign
> line terminators to creep-in into the equation in one section
> of the resulting file.

The integration of altermime into Amavis needs some adjustment to handle
the conflicting behaviour. If Amavis expects an LF-terminated output file
from altermime, it should ensure that this is the case by stripping any
extraneous CR characters at the ends of lines.

It is the DKIM signer's job to ensure the content it is signing is
minimally canonical. You could encode the content as QP if you felt
it is not your job to modify the content produced by altermime even in
this (bug) case.

--
        Viktor.

Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.

To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:[hidden email]?body=unsubscribe%20postfix-users>

If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.
Reply | Threaded
Open this post in threaded view
|

Re: dkim + altermime disclaimer

mouss-2
In reply to this post by Mark Martinec-5
Mark Martinec wrote:

> mouss,
>
>  
>>> When altermime is inserting a '--disclaimer-html' into a html mail,
>>> for some reason it inserts a CR before each LF of the inserted html text,
>>>      
>> so it sends lines ending in CRLF? if so, this looks ok to me. or am I
>> misunderstanding?
>>    
>
> Just to clarify: altermime is dealing with plain text files
> (it doesn't transfer anything by itself), i.e. with native
> Unix line-endings text files. An original mail and a disclaimer
> text reside on their own native text files, and the resulting mail
> ends up as a native Unix text file. There is no place for foreign
> line terminators to creep-in into the equation in one section
> of the resulting file.
>
>  
>> the question is why this becomes a CRCRLF when it was simply a CRLF?
>>    
>
> SMTP mandates that line endings should look on the wire as a CRLF pair.
> So a native end-of-line (NL) is translated into CRLF,

how about replacing \r*\n with \r\n ?

>  and a standalone
> extra CR is kept unmodified.
>  

A bare CR is problematic: if it is "fixed" by an intermediary MTA, the
signature breaks.