OpenSSL 1.0.1g and Ironport SMTP appliances interop issue

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

OpenSSL 1.0.1g and Ironport SMTP appliances interop issue

Viktor Dukhovni

[ Original openssl-users thread subject:
  openssl update 1.0.1f to 1.0.1g broke sendmail ... ]

In a thread on the openssl-users list there is a report of an
upgrade to OpenSSL 1.0.1g (to deal with "Heartbleed") causing one
Sendmail system delivery problems to a few sites.  This is more
noticeable with Sendmail, because Sendmail (in at least some
installations) does not reconnect and retry in cleartext after a
STARTTLS handshake fails.

The problem is caused by the fact that 1.0.1g includes more than
just the "Heartbleed" fix, there is a also a new work-around for
TLSv1.2 interoperability issues with some older (but not uncommon)
F5 load-balancers that are popular as SSL termination devices for
busy websites.

    http://tools.ietf.org/html/draft-agl-tls-padding-03

This work-around avoids sending SSL client HELLO messages whose
size is in the range from 256 to 511 bytes.  This size range causes
the F5 to misinterpret the message as a truncated SSLv2 HELLO rather
than a valid SSLv3 HELLO.  The 1.0.1g work-around sends a new TLS
extension that contains enough zeros to pad the client HELLO to at
least 512 bytes when it would otherwise be longer than 256 bytes.

Unfortunately, it seems that this work-around causes problems with
some Ironport email appliances.  I've been able to reproduce the
problem with Postfix linked with OpenSSL 1.0.1g.  The real issue
is in Ironport's TLS stack and has little to do with Sendmail or
Postfix.  It is difficult to predict how long it will take Ironport
to deploy a fix to affected customers.  Clearly these systems are
now going to have some trouble receiving (TLS) email.

I imagine that some of you will see similar problems, the symptoms
in the logs will look something like this:

    postfix/smtp[<pid>]: warning: TLS library problem:
        error:1407741A: SSL routines:
        SSL23_GET_SERVER_HELLO:tlsv1 alert decode error:
        s23_clnt.c:762:

the important part being:

    "SSL23_GET_SERVER_HELLO:tlsv1 alert decode error".

The only way to work-around this with Postfix linked to OpenSSL
1.0.1g and continue to encrypt traffic to the destinations in
question is to force the use of SSLv3 only.  This requires
a compatible Postfix version:

    * >= 2.6.15 if 2.6.x
    * >= 2.7.9 if 2.7.x
    * >= 2.8.10 if 2.8.x
    * >= 2.9.2 if 2.9.x
    * 2.10.0 and up

  tls_policy:
    example.com may protocols=SSLv3
    example.org encrypt protocols=SSLv3
    example.org fingerprint protocols=SSLv3 match=...
    example.org secure protocols=SSLv3

Note that various vendor SSL updates for "Heartbleed" may not
exhibit the issue.  For example, Debian wheezy back-ported just the
relevant bug-fix to without back-porting the new padding extension.
I also expect similar (fortunate) behaviour on system's with OpenSSL
patched by RedHat, and various others.

Other than re-compiling OpenSSL to exclude the new feature. forcing
SSLv3 is the only way to disable it in stock 1.0.1g.

The git commits that introduced the new feature are below my
signature.  If you build OpenSSL from source and use the software
primarily for SMTP, you may want to modify the code to disable the
new extension, e.g. by commenting out the line:

    #define TLSEXT_TYPE_padding    21

in ssl/tls1.h.  Of course then interoperability with F5 SSL
terminating load-balancers will not be improved.  I don't know
whether the original F5 issue was ever a problem for SMTP, it is
AFAIK primarily a browser to webserver issue.

--
        Viktor.

commit 4a55631e4dc76fb8d668218bf461c45a9abc5b94
Author: Dr. Stephen Henson <[hidden email]>
Date:   Fri Dec 13 14:41:32 2013 +0000

    Backport TLS padding extension from master.
    (cherry picked from commit 8c6d8c2a498146992123ef5407d7ba01a1e7224d)
   
    Conflicts:
   
    CHANGES
    ssl/t1_lib.c

diff --git a/CHANGES b/CHANGES
index f6fabf9..58ac884 100644
--- a/CHANGES
+++ b/CHANGES
@@ -4,7 +4,24 @@
 
  Changes between 1.0.1f and 1.0.1g [xx XXX xxxx]
 
-  *)
+  *) TLS pad extension: draft-agl-tls-padding-02
+
+     Workaround for the "TLS hang bug" (see FAQ and PR#2771): if the
+     TLS client Hello record length value would otherwise be > 255 and
+     less that 512 pad with a dummy extension containing zeroes so it
+     is at least 512 bytes long.
+
+     To enable it use an unused extension number (for example chrome uses
+     35655) using:
+
+     e.g. -DTLSEXT_TYPE_padding=35655
+
+     Since the extension is ignored the actual number doesn't matter as long
+     as it doesn't clash with any existing extension.
+
+     This will be updated when the extension gets an official number.
+
+     [Adam Langley, Steve Henson]
 
  Changes between 1.0.1e and 1.0.1f [6 Jan 2014]
 
diff --git a/ssl/t1_lib.c b/ssl/t1_lib.c
index e22ebbf..29ccd83 100644
--- a/ssl/t1_lib.c
+++ b/ssl/t1_lib.c
@@ -662,6 +662,36 @@ unsigned char *ssl_add_clienthello_tlsext(SSL *s, unsigned char *p, unsigned cha
                 }
 #endif
 
+#ifdef TLSEXT_TYPE_padding
+ /* Add padding to workaround bugs in F5 terminators.
+ * See https://tools.ietf.org/html/draft-agl-tls-padding-02
+ *
+ * NB: because this code works out the length of all existing
+ * extensions it MUST always appear last.
+ */
+ {
+ int hlen = ret - (unsigned char *)s->init_buf->data;
+ /* The code in s23_clnt.c to build ClientHello messages includes the
+ * 5-byte record header in the buffer, while the code in s3_clnt.c does
+ * not. */
+ if (s->state == SSL23_ST_CW_CLNT_HELLO_A)
+ hlen -= 5;
+ if (hlen > 0xff && hlen < 0x200)
+ {
+ hlen = 0x200 - hlen;
+ if (hlen >= 4)
+ hlen -= 4;
+ else
+ hlen = 0;
+
+ s2n(TLSEXT_TYPE_padding, ret);
+ s2n(hlen, ret);
+ memset(ret, 0, hlen);
+ ret += hlen;
+ }
+ }
+#endif
+
  if ((extdatalen = ret-p-2)== 0)
  return p;
 
commit 51624dbdaed5325ac763e63dc5eb0b3ef85d6489
Author: Dr. Stephen Henson <[hidden email]>
Date:   Sat Apr 5 20:43:54 2014 +0100

    Set TLS padding extension value.
   
    Enable TLS padding extension using official value from:
   
    http://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml
    (cherry picked from commit cd6bd5ffda616822b52104fee0c4c7d623fd4f53)
   
    Conflicts:
   
    CHANGES
    ssl/tls1.h

diff --git a/CHANGES b/CHANGES
index 99aeefb..0484456 100644
--- a/CHANGES
+++ b/CHANGES
@@ -13,23 +13,13 @@
      flaw and to Yuval Yarom for supplying a fix (CVE-2014-0076)
      [Yuval Yarom and Naomi Benger]
 
-  *) TLS pad extension: draft-agl-tls-padding-02
+  *) TLS pad extension: draft-agl-tls-padding-03
 
      Workaround for the "TLS hang bug" (see FAQ and PR#2771): if the
      TLS client Hello record length value would otherwise be > 255 and
      less that 512 pad with a dummy extension containing zeroes so it
      is at least 512 bytes long.
 
-     To enable it use an unused extension number (for example chrome uses
-     35655) using:
-
-     e.g. -DTLSEXT_TYPE_padding=35655
-
-     Since the extension is ignored the actual number doesn't matter as long
-     as it doesn't clash with any existing extension.
-
-     This will be updated when the extension gets an official number.
-
      [Adam Langley, Steve Henson]
 
  Changes between 1.0.1e and 1.0.1f [6 Jan 2014]
diff --git a/ssl/t1_lib.c b/ssl/t1_lib.c
index 29ccd83..b82fada 100644
--- a/ssl/t1_lib.c
+++ b/ssl/t1_lib.c
@@ -664,7 +664,7 @@ unsigned char *ssl_add_clienthello_tlsext(SSL *s, unsigned char *p, unsigned cha
 
 #ifdef TLSEXT_TYPE_padding
  /* Add padding to workaround bugs in F5 terminators.
- * See https://tools.ietf.org/html/draft-agl-tls-padding-02
+ * See https://tools.ietf.org/html/draft-agl-tls-padding-03
  *
  * NB: because this code works out the length of all existing
  * extensions it MUST always appear last.
diff --git a/ssl/tls1.h b/ssl/tls1.h
index c39c267..c992091 100644
--- a/ssl/tls1.h
+++ b/ssl/tls1.h
@@ -230,6 +230,12 @@ extern "C" {
 /* ExtensionType value from RFC5620 */
 #define TLSEXT_TYPE_heartbeat 15
 
+/* ExtensionType value for TLS padding extension.
+ * http://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml
+ * http://tools.ietf.org/html/draft-agl-tls-padding-03
+ */
+#define TLSEXT_TYPE_padding 21
+
 /* ExtensionType value from RFC4507 */
 #define TLSEXT_TYPE_session_ticket 35
 
Reply | Threaded
Open this post in threaded view
|

Re: OpenSSL 1.0.1g and Ironport SMTP appliances interop issue

lists@rhsoft.net

Am 11.04.2014 06:53, schrieb Viktor Dukhovni:
> Note that various vendor SSL updates for "Heartbleed" may not
> exhibit the issue.  For example, Debian wheezy back-ported just the
> relevant bug-fix to without back-porting the new padding extension.
> I also expect similar (fortunate) behaviour on system's with OpenSSL
> patched by RedHat, and various others

confirmed for RedHat, at least i expect the same
changeset for RHEL and Fedora

https://koji.fedoraproject.org/koji/packageinfo?packageID=109
openssl-1.0.1e-37.fc19.1 / openssl-1.0.1e-37.fc20.1

* Mon Apr 07 2014 Dennis Gilmore <[hidden email]>
- 1.0.1e-37.1 - pull in upstream patch for CVE-2014-0160
- removed CHANGES file portion from patch for expediency

* Tue Jan 07 2014 Tomáš Mráz <[hidden email]> 1.0.1e-37
- fix CVE-2013-4353
- Invalid TLS handshake crash
- fix CVE-2013-6450 - possible MiTM attack on DTLS1