OpenSSL 1.0.1g and Ironport SMTP appliances interop issue
[ Original openssl-users thread subject:
openssl update 1.0.1f to 1.0.1g broke sendmail ... ]
In a thread on the openssl-users list there is a report of an
upgrade to OpenSSL 1.0.1g (to deal with "Heartbleed") causing one
Sendmail system delivery problems to a few sites. This is more
noticeable with Sendmail, because Sendmail (in at least some
installations) does not reconnect and retry in cleartext after a
STARTTLS handshake fails.
The problem is caused by the fact that 1.0.1g includes more than
just the "Heartbleed" fix, there is a also a new work-around for
TLSv1.2 interoperability issues with some older (but not uncommon)
F5 load-balancers that are popular as SSL termination devices for
This work-around avoids sending SSL client HELLO messages whose
size is in the range from 256 to 511 bytes. This size range causes
the F5 to misinterpret the message as a truncated SSLv2 HELLO rather
than a valid SSLv3 HELLO. The 1.0.1g work-around sends a new TLS
extension that contains enough zeros to pad the client HELLO to at
least 512 bytes when it would otherwise be longer than 256 bytes.
Unfortunately, it seems that this work-around causes problems with
some Ironport email appliances. I've been able to reproduce the
problem with Postfix linked with OpenSSL 1.0.1g. The real issue
is in Ironport's TLS stack and has little to do with Sendmail or
Postfix. It is difficult to predict how long it will take Ironport
to deploy a fix to affected customers. Clearly these systems are
now going to have some trouble receiving (TLS) email.
I imagine that some of you will see similar problems, the symptoms
in the logs will look something like this:
The only way to work-around this with Postfix linked to OpenSSL
1.0.1g and continue to encrypt traffic to the destinations in
question is to force the use of SSLv3 only. This requires
a compatible Postfix version:
* >= 2.6.15 if 2.6.x
* >= 2.7.9 if 2.7.x
* >= 2.8.10 if 2.8.x
* >= 2.9.2 if 2.9.x
* 2.10.0 and up
Note that various vendor SSL updates for "Heartbleed" may not
exhibit the issue. For example, Debian wheezy back-ported just the
relevant bug-fix to without back-porting the new padding extension.
I also expect similar (fortunate) behaviour on system's with OpenSSL
patched by RedHat, and various others.
Other than re-compiling OpenSSL to exclude the new feature. forcing
SSLv3 is the only way to disable it in stock 1.0.1g.
The git commits that introduced the new feature are below my
signature. If you build OpenSSL from source and use the software
primarily for SMTP, you may want to modify the code to disable the
new extension, e.g. by commenting out the line:
#define TLSEXT_TYPE_padding 21
in ssl/tls1.h. Of course then interoperability with F5 SSL
terminating load-balancers will not be improved. I don't know
whether the original F5 issue was ever a problem for SMTP, it is
AFAIK primarily a browser to webserver issue.
Author: Dr. Stephen Henson <[hidden email]>
Date: Fri Dec 13 14:41:32 2013 +0000
Backport TLS padding extension from master.
(cherry picked from commit 8c6d8c2a498146992123ef5407d7ba01a1e7224d)
+ *) TLS pad extension: draft-agl-tls-padding-02
+ Workaround for the "TLS hang bug" (see FAQ and PR#2771): if the
+ TLS client Hello record length value would otherwise be > 255 and
+ less that 512 pad with a dummy extension containing zeroes so it
+ is at least 512 bytes long.
+ To enable it use an unused extension number (for example chrome uses
+ 35655) using:
+ e.g. -DTLSEXT_TYPE_padding=35655
+ Since the extension is ignored the actual number doesn't matter as long
+ as it doesn't clash with any existing extension.
+ This will be updated when the extension gets an official number.
+ [Adam Langley, Steve Henson]
+ /* Add padding to workaround bugs in F5 terminators.
+ * See https://tools.ietf.org/html/draft-agl-tls-padding-02 + *
+ * NB: because this code works out the length of all existing
+ * extensions it MUST always appear last.
+ int hlen = ret - (unsigned char *)s->init_buf->data;
+ /* The code in s23_clnt.c to build ClientHello messages includes the
+ * 5-byte record header in the buffer, while the code in s3_clnt.c does
+ * not. */
+ if (s->state == SSL23_ST_CW_CLNT_HELLO_A)
+ hlen -= 5;
+ if (hlen > 0xff && hlen < 0x200)
+ hlen = 0x200 - hlen;
+ if (hlen >= 4)
+ hlen -= 4;
+ hlen = 0;
+ s2n(TLSEXT_TYPE_padding, ret);
+ s2n(hlen, ret);
+ memset(ret, 0, hlen);
+ ret += hlen;
if ((extdatalen = ret-p-2)== 0)
Author: Dr. Stephen Henson <[hidden email]>
Date: Sat Apr 5 20:43:54 2014 +0100
Set TLS padding extension value.
Enable TLS padding extension using official value from:
diff --git a/CHANGES b/CHANGES
index 99aeefb..0484456 100644
@@ -13,23 +13,13 @@
flaw and to Yuval Yarom for supplying a fix (CVE-2014-0076)
[Yuval Yarom and Naomi Benger]
- *) TLS pad extension: draft-agl-tls-padding-02
+ *) TLS pad extension: draft-agl-tls-padding-03
Workaround for the "TLS hang bug" (see FAQ and PR#2771): if the
TLS client Hello record length value would otherwise be > 255 and
less that 512 pad with a dummy extension containing zeroes so it
is at least 512 bytes long.
- To enable it use an unused extension number (for example chrome uses
- 35655) using:
- e.g. -DTLSEXT_TYPE_padding=35655
- Since the extension is ignored the actual number doesn't matter as long
- as it doesn't clash with any existing extension.
- This will be updated when the extension gets an official number.
[Adam Langley, Steve Henson]
Changes between 1.0.1e and 1.0.1f [6 Jan 2014]
diff --git a/ssl/t1_lib.c b/ssl/t1_lib.c
index 29ccd83..b82fada 100644
@@ -664,7 +664,7 @@ unsigned char *ssl_add_clienthello_tlsext(SSL *s, unsigned char *p, unsigned cha
Re: OpenSSL 1.0.1g and Ironport SMTP appliances interop issue
Am 11.04.2014 06:53, schrieb Viktor Dukhovni:
> Note that various vendor SSL updates for "Heartbleed" may not
> exhibit the issue. For example, Debian wheezy back-ported just the
> relevant bug-fix to without back-porting the new padding extension.
> I also expect similar (fortunate) behaviour on system's with OpenSSL
> patched by RedHat, and various others
confirmed for RedHat, at least i expect the same
changeset for RHEL and Fedora