Fwd: Bug#924183: postfix: Trust anchor files (tafile=) in TLS policy break secure level email delivery

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Fwd: Bug#924183: postfix: Trust anchor files (tafile=) in TLS policy break secure level email delivery

Scott Kitterman-4
I received the bug report/patch below from a Debian user.  I'm somewhat busy this weekend/week, so I decided to forward it without evaluation rather than sit on it for a week until I could research it.

I attempted to remove the distro specific noise from the report.

Scott K



Package: postfix
Version: 3.4.1-1
Severity: important
Tags: patch

Hi,

I have entries like this:

  [domain.tld]:587    secure tafile=/etc/ssl/certs/Lets-Encrypt-Authority-X3.pem

… in the file referenced by:

  smtp_tls_policy_maps = hash:/etc/postfix/tls_policy

This worked just fine until 3.3.2-4 inclusive but since I've upgraded
my sid system yesterday and Postfix was upgraded to 3.4.1-1 I see:

  postfix/smtp[15202]: warning: Trust anchor files not supported
  postfix/smtp[15202]: warning: TLS policy lookup error for [domain.tld]:587/domain.tld: client TLS configuration problem
  postfix/smtp[15202]: warning: TLS policy lookup for [domain.tld]:587/domain.tld: client TLS configuration problem
  postfix/smtp[15202]: 8B30018835E3: to=<[hidden email]>, relay=none, delay=1197, delays=1196/0.82/0.36/0, dsn=4.7.5, status=deferred (client TLS configuration problem)

This seems to come from src/tls/tls_dane.c. I see that 3.4.0 has
modified this file quite a bit, e.g. these lines were removed:

  #if OPENSSL_VERSION_NUMBER >= 0x1000000fL && \
         (defined(X509_V_FLAG_PARTIAL_CHAIN) || !defined(OPENSSL_NO_ECDH))
  #define TRUST_ANCHOR_SUPPORT

… and there's only one "#ifdef TRUST_ANCHOR_SUPPORT" left, that guards
the warning I'm seeing. This feels like a leftover of an incomplete
cleanup of the TLS support code that happened in this release, such as
dropping support for OpenSSL 1.0.1.

FWIW the attached patch fixes this problem for me. I don't know if it
can cause any trouble.


0001-Drop-leftover-of-obsolete-check-for-trust-anchor-sup.patch (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Bug#924183: postfix: Trust anchor files (tafile=) in TLS policy break secure level email delivery

Wietse Venema
Scott Kitterman:
> I received the bug report/patch below from a Debian user.  I'm somewhat busy this weekend/week, so I decided to forward it without evaluation rather than sit on it for a week until I could research it.
>
> I attempted to remove the distro specific noise from the report.

Sorry for making you the guinea pig.

I am considering to withdraw Postfix 3.4 and do a proper Postfix
3.5 release as planned later this year. We can't afford having a
stable release with bug-of-the-week fixes like we have now.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Bug#924183: postfix: Trust anchor files (tafile=) in TLS policy break secure level email delivery

Scott Kitterman-4
On Sunday, March 10, 2019 11:11:15 AM Wietse Venema wrote:

> Scott Kitterman:
> > I received the bug report/patch below from a Debian user.  I'm somewhat
> > busy this weekend/week, so I decided to forward it without evaluation
> > rather than sit on it for a week until I could research it.
> >
> > I attempted to remove the distro specific noise from the report.
>
> Sorry for making you the guinea pig.
>
> I am considering to withdraw Postfix 3.4 and do a proper Postfix
> 3.5 release as planned later this year. We can't afford having a
> stable release with bug-of-the-week fixes like we have now.

I knew I was taking a risk jumping to 3.4 late in our release cycle.

A week from now when it hits Debian Testing, the user base will grow
significantly and we'll get more feedback.

I guess there weren't enough testers for 3.4 before release.  I don't know
that that situation will be better later in the year for 3.5.  From my point
of view (I don't know about other distros/OS), it would be somewhat painful to
stay on 3.3 for the next release at this point, but it'll be a lot harder a
week from now.

My preference would be to press on with 3.4 (I don't mind packaging the bug
fixes if you don't mind releasing them), but if you are going to withdraw 3.4,
please do it before next Sunday so I can keep it out of the next Debian
release.

Although Debian doesn't normally allow it, I've gotten permission from the
release team to update postfix based on your microreleases (thir digit) based
on the good history with them fixing relevant bugs with minimal regression
risk.  It's absolutely not a problem for me to stay on the 3.4 path if you're
up for it.

Scott K
Reply | Threaded
Open this post in threaded view
|

Re: Bug#924183: postfix: Trust anchor files (tafile=) in TLS policy break secure level email delivery

Postfix User-2
In reply to this post by Wietse Venema
On Sun, 10 Mar 2019 11:11:15 -0400 (EDT), Wietse Venema stated:

>Scott Kitterman:
>> I received the bug report/patch below from a Debian user.  I'm
>> somewhat busy this weekend/week, so I decided to forward it without
>> evaluation rather than sit on it for a week until I could research
>> it.
>>
>> I attempted to remove the distro specific noise from the report.  
>
>Sorry for making you the guinea pig.
>
>I am considering to withdraw Postfix 3.4 and do a proper Postfix
>3.5 release as planned later this year. We can't afford having a
>stable release with bug-of-the-week fixes like we have now.
>
> Wietse


Wietse, the number of bugs found in stable releases of Postfix is
infinitesimal when compared to other vendors who release products under
the "fix it later" theory.

--
Postfix User
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Bug#924183: postfix: Trust anchor files (tafile=) in TLS policy break secure level email delivery

pgndev
In reply to this post by Scott Kitterman-4
To add a possible data point to the convo,

at least one distro, OpenSUSE, is already toying with apparently poorly
thought-thru patches (aka, not vetted/source here, from upstream) --
e,g, here,
https://build.opensuse.org/package/view_file/openSUSE:Factory/postfix/postfix-linux45.patch?expand=1 
-- to 'old release' postfix 3.3.3 for Linux5 support.

no particular skin off _my_ nose -- i contentedly build/deploy only from
Postfix upstream source.

i understand it's 'their' decision, good or bad.  nonetheless, it will
inevitably generate noise here.

Reply | Threaded
Open this post in threaded view
|

Re: Bug#924183: postfix: Trust anchor files (tafile=) in TLS policy break secure level email delivery

Postfix User-2
In reply to this post by Scott Kitterman-4
On Sun, 10 Mar 2019 11:41:01 -0400, Scott Kitterman stated:

>On Sunday, March 10, 2019 11:11:15 AM Wietse Venema wrote:
>> Scott Kitterman:  
>> > I received the bug report/patch below from a Debian user.  I'm
>> > somewhat busy this weekend/week, so I decided to forward it
>> > without evaluation rather than sit on it for a week until I could
>> > research it.
>> >
>> > I attempted to remove the distro specific noise from the report.  
>>
>> Sorry for making you the guinea pig.
>>
>> I am considering to withdraw Postfix 3.4 and do a proper Postfix
>> 3.5 release as planned later this year. We can't afford having a
>> stable release with bug-of-the-week fixes like we have now.  
>
>I knew I was taking a risk jumping to 3.4 late in our release cycle.
>
>A week from now when it hits Debian Testing, the user base will grow
>significantly and we'll get more feedback.
>
>I guess there weren't enough testers for 3.4 before release.  I don't
>know that that situation will be better later in the year for 3.5.
>From my point of view (I don't know about other distros/OS), it would
>be somewhat painful to stay on 3.3 for the next release at this point,
>but it'll be a lot harder a week from now.
>
>My preference would be to press on with 3.4 (I don't mind packaging
>the bug fixes if you don't mind releasing them), but if you are going
>to withdraw 3.4, please do it before next Sunday so I can keep it out
>of the next Debian release.
>
>Although Debian doesn't normally allow it, I've gotten permission from
>the release team to update postfix based on your microreleases (thir
>digit) based on the good history with them fixing relevant bugs with
>minimal regression risk.  It's absolutely not a problem for me to stay
>on the 3.4 path if you're up for it.
>
>Scott K

FreeBSD is using Postfix 3.3 for its stable release and Postfix 3.4 for
its development (experimental) release.

--
Postfix User
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Bug#924183: postfix: Trust anchor files (tafile=) in TLS policy break secure level email delivery

Wietse Venema
In reply to this post by Scott Kitterman-4
Scott Kitterman:

> On Sunday, March 10, 2019 11:11:15 AM Wietse Venema wrote:
> > Scott Kitterman:
> > > I received the bug report/patch below from a Debian user.  I'm somewhat
> > > busy this weekend/week, so I decided to forward it without evaluation
> > > rather than sit on it for a week until I could research it.
> > >
> > > I attempted to remove the distro specific noise from the report.
> >
> > Sorry for making you the guinea pig.
> >
> > I am considering to withdraw Postfix 3.4 and do a proper Postfix
> > 3.5 release as planned later this year. We can't afford having a
> > stable release with bug-of-the-week fixes like we have now.
>
> I knew I was taking a risk jumping to 3.4 late in our release cycle.
>
> A week from now when it hits Debian Testing, the user base will grow
> significantly and we'll get more feedback.
>
> I guess there weren't enough testers for 3.4 before release.  I don't know
> that that situation will be better later in the year for 3.5.  From my point
> of view (I don't know about other distros/OS), it would be somewhat painful to
> stay on 3.3 for the next release at this point, but it'll be a lot harder a
> week from now.
>
> My preference would be to press on with 3.4 (I don't mind packaging the bug
> fixes if you don't mind releasing them), but if you are going to withdraw 3.4
> please do it before next Sunday so I can keep it out of the next Debian
> release.

We know of multiple bugs that broke 'desirable functionality' after
an overhaul of the TLS stack, and that were kindly brought to the
developer's attention by folks like you.

I have to consider the possibility that the same overhaul introduced
an equal if not larger number of bugs with 'undesirable functionality',
and that these bugs will be found by not-so-kind folks, who will
report them only if it helps to promote themselves while at the
same time destroying Postfix's good reputation.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Bug#924183: postfix: Trust anchor files (tafile=) in TLS policy break secure level email delivery

Viktor Dukhovni
In reply to this post by Scott Kitterman-4
On Sun, Mar 10, 2019 at 02:34:02PM +0000, Scott Kitterman wrote:

> This worked just fine until 3.3.2-4 inclusive but since I've upgraded
> my sid system yesterday and Postfix was upgraded to 3.4.1-1 I see:
>
>   postfix/smtp[15202]: warning: Trust anchor files not supported
>   postfix/smtp[15202]: warning: TLS policy lookup error for [domain.tld]:587/domain.tld: client TLS configuration problem
>   postfix/smtp[15202]: warning: TLS policy lookup for [domain.tld]:587/domain.tld: client TLS configuration problem
>   postfix/smtp[15202]: 8B30018835E3: to=<[hidden email]>, relay=none, delay=1197, delays=1196/0.82/0.36/0, dsn=4.7.5, status=deferred (client TLS configuration problem)

> diff --git a/src/tls/tls_dane.c b/src/tls/tls_dane.c
> index 93f8e2a5..013426b1 100644
> --- a/src/tls/tls_dane.c
> +++ b/src/tls/tls_dane.c
> @@ -1125,7 +1125,6 @@ TLS_DANE *tls_dane_resolve(unsigned port, const char *proto, DNS_RR *hostrr,
>  
>  int     tls_dane_load_trustfile(TLS_DANE *dane, const char *tafile)
>  {
> -#ifdef TRUST_ANCHOR_SUPPORT
>      BIO    *bp;
>      char   *name = 0;
>      char   *header = 0;
> @@ -1217,9 +1216,6 @@ int     tls_dane_load_trustfile(TLS_DANE *dane, const char *tafile)
>      }
>      /* Some other PEM read error */
>      tls_print_errors();
> -#else
> -    msg_warn("Trust anchor files not supported");
> -#endif
>      return (0);
>  }

The proposed patch is correct.  Repeated below without "quoting":

diff --git a/src/tls/tls_dane.c b/src/tls/tls_dane.c
index 93f8e2a5..013426b1 100644
--- a/src/tls/tls_dane.c
+++ b/src/tls/tls_dane.c
@@ -1125,7 +1125,6 @@ TLS_DANE *tls_dane_resolve(unsigned port, const char *proto, DNS_RR *hostrr,
 
 int     tls_dane_load_trustfile(TLS_DANE *dane, const char *tafile)
 {
-#ifdef TRUST_ANCHOR_SUPPORT
     BIO    *bp;
     char   *name = 0;
     char   *header = 0;
@@ -1217,9 +1216,6 @@ int     tls_dane_load_trustfile(TLS_DANE *dane, const char *tafile)
     }
     /* Some other PEM read error */
     tls_print_errors();
-#else
-    msg_warn("Trust anchor files not supported");
-#endif
     return (0);
 }
 

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Bug#924183: postfix: Trust anchor files (tafile=) in TLS policy break secure level email delivery

Viktor Dukhovni
In reply to this post by Wietse Venema
On Sun, Mar 10, 2019 at 12:29:44PM -0400, Wietse Venema wrote:

> > My preference would be to press on with 3.4 (I don't mind packaging the bug
> > fixes if you don't mind releasing them), but if you are going to withdraw 3.4
> > please do it before next Sunday so I can keep it out of the next Debian
> > release.
>
> We know of multiple bugs that broke 'desirable functionality' after
> an overhaul of the TLS stack, and that were kindly brought to the
> developer's attention by folks like you.
>
> I have to consider the possibility that the same overhaul introduced
> an equal if not larger number of bugs with 'undesirable functionality',
> and that these bugs will be found by not-so-kind folks, who will
> report them only if it helps to promote themselves while at the
> same time destroying Postfix's good reputation.

Some of the changes in 3.4 are quite useful when Postfix is compiled
with OpenSSL 1.1.1, which introduces TLS 1.3.

My sense is that the situation is not so dire as to warrant retracting
the release.  The recent changes in 3.4 are:

    - Support for server-side SNI, with new code to load multi-algorithm
      certificate chain sets.  This also loads the key + cert in a
      single pass when both are in the same file, now with the correct
      test for the return value (first report bug) and tests.

    - More diligent tlsproxy(8) checking for compatible parameters in TLS
      connection re-use.  The tlsproxy connection re-use was added earlier,
      and the more recent changes just fine-tune context sharing.

    - Removal of support for OpenSSL 1.0.2.  Reduces the attack
      surface and intoduces no new code.

      This was however the source of the present (second) reported
      bug, because an #ifdef guard testing for 1.0.2 or later that
      should have been removed, was left in place.

    - Consolidation of TLS summary logging, and better logging of
      TLS 1.3 handshake properties.

Looking over the various changes again, I think 3.4 is fine.  I'll
do another code review this week.

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Bug#924183: postfix: Trust anchor files (tafile=) in TLS policy break secure level email delivery

Wietse Venema
Viktor Dukhovni:

> On Sun, Mar 10, 2019 at 12:29:44PM -0400, Wietse Venema wrote:
>
> > > My preference would be to press on with 3.4 (I don't mind packaging the bug
> > > fixes if you don't mind releasing them), but if you are going to withdraw 3.4
> > > please do it before next Sunday so I can keep it out of the next Debian
> > > release.
> >
> > We know of multiple bugs that broke 'desirable functionality' after
> > an overhaul of the TLS stack, and that were kindly brought to the
> > developer's attention by folks like you.
> >
> > I have to consider the possibility that the same overhaul introduced
> > an equal if not larger number of bugs with 'undesirable functionality',
> > and that these bugs will be found by not-so-kind folks, who will
> > report them only if it helps to promote themselves while at the
> > same time destroying Postfix's good reputation.
>
> Some of the changes in 3.4 are quite useful when Postfix is compiled
> with OpenSSL 1.1.1, which introduces TLS 1.3.
>
> My sense is that the situation is not so dire as to warrant retracting
> the release.  The recent changes in 3.4 are:
>
>     - Support for server-side SNI, with new code to load multi-algorithm
>       certificate chain sets.  This also loads the key + cert in a
>       single pass when both are in the same file, now with the correct
>       test for the return value (first report bug) and tests.
>
>     - More diligent tlsproxy(8) checking for compatible parameters in TLS
>       connection re-use.  The tlsproxy connection re-use was added earlier,
>       and the more recent changes just fine-tune context sharing.
>
>     - Removal of support for OpenSSL 1.0.2.  Reduces the attack
>       surface and intoduces no new code.
>
>       This was however the source of the present (second) reported
>       bug, because an #ifdef guard testing for 1.0.2 or later that
>       should have been removed, was left in place.
>
>     - Consolidation of TLS summary logging, and better logging of
>       TLS 1.3 handshake properties.
>
> Looking over the various changes again, I think 3.4 is fine.  I'll
> do another code review this week.

You are looking from the "we made improvements" angle. I am looking
from the "with hard work, we introduce 1 bug in 1000 lines of new
code" angle.

In the TLS library there were 1039 additions and 559 deletions from
Postfix 3.3.3 to 3.4.1 (diff -bur --new-file for 'c' and 'h' files
only, excluding proxy-related code). That count does not distinguish
between low-impact changes that affect only nearby code, and
high-impact changes that affect multiple lines, such as global ifdefs.

The changes among those 1600 lines that 'broke' intended functionality
are 'easy' to weed out - just wait for people to to report breakages.
Such a reactive approach might be acceptable in some projects.

I am concerned about the changes among those 1600 lines that did
NOT break intended behavior, but that introduced some other problem.

What is the basis for confidence that no such problems have been
introduced, if we obviously missed multiple things that could have
been found with simple tests?

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Bug#924183: postfix: Trust anchor files (tafile=) in TLS policy break secure level email delivery

Daniele Nicolodi
On 10/03/2019 15:07, Wietse Venema wrote:

> You are looking from the "we made improvements" angle. I am looking
> from the "with hard work, we introduce 1 bug in 1000 lines of new
> code" angle.
>
> In the TLS library there were 1039 additions and 559 deletions from
> Postfix 3.3.3 to 3.4.1 (diff -bur --new-file for 'c' and 'h' files
> only, excluding proxy-related code). That count does not distinguish
> between low-impact changes that affect only nearby code, and
> high-impact changes that affect multiple lines, such as global ifdefs.
>
> The changes among those 1600 lines that 'broke' intended functionality
> are 'easy' to weed out - just wait for people to to report breakages.
> Such a reactive approach might be acceptable in some projects.
>
> I am concerned about the changes among those 1600 lines that did
> NOT break intended behavior, but that introduced some other problem.
>
> What is the basis for confidence that no such problems have been
> introduced, if we obviously missed multiple things that could have
> been found with simple tests?

I have no say in the development of Postfix, however I believe another
interesting question that should be asked is: what is going to make you
more confident in the releasing those changes later this year with
Postfix 3.5?  Are there circumstances for which more testing and code
auditing will happen if the code is not released?

Cheers,
Dan
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Bug#924183: postfix: Trust anchor files (tafile=) in TLS policy break secure level email delivery

Wietse Venema
Daniele Nicolodi:

> On 10/03/2019 15:07, Wietse Venema wrote:
> > You are looking from the "we made improvements" angle. I am looking
> > from the "with hard work, we introduce 1 bug in 1000 lines of new
> > code" angle.
> >
> > In the TLS library there were 1039 additions and 559 deletions from
> > Postfix 3.3.3 to 3.4.1 (diff -bur --new-file for 'c' and 'h' files
> > only, excluding proxy-related code). That count does not distinguish
> > between low-impact changes that affect only nearby code, and
> > high-impact changes that affect multiple lines, such as global ifdefs.
> >
> > The changes among those 1600 lines that 'broke' intended functionality
> > are 'easy' to weed out - just wait for people to to report breakages.
> > Such a reactive approach might be acceptable in some projects.
> >
> > I am concerned about the changes among those 1600 lines that did
> > NOT break intended behavior, but that introduced some other problem.
> >
> > What is the basis for confidence that no such problems have been
> > introduced, if we obviously missed multiple things that could have
> > been found with simple tests?
>
> I have no say in the development of Postfix, however I believe another
> interesting question that should be asked is: what is going to make you
> more confident in the releasing those changes later this year with
> Postfix 3.5?  Are there circumstances for which more testing and code
> auditing will happen if the code is not released?

Because we won't be changing 1600 lines in a critical library two
months before the release.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Bug#924183: postfix: Trust anchor files (tafile=) in TLS policy break secure level email delivery

Lex Scarisbrick
In reply to this post by Scott Kitterman-4
On Sun, Mar 10, 2019 at 9:41 AM Scott Kitterman <[hidden email]> wrote:
> My preference would be to press on with 3.4 (I don't mind packaging the bug
> fixes if you don't mind releasing them), but if you are going to withdraw 3.4,
> please do it before next Sunday so I can keep it out of the next Debian
> release.

+1

The RFC 3030 support in Postfix 3.4 is very much a welcome addition for interoperability with Microsoft.  I understand that's coming at it from wanting something additive and not about what might be broken as a result.  I'm happy to help if there's any specific testing that would help stabilize the release.  FWIW, I haven't seen any issues ingressing mail from Microsoft with Postfix 3.4 to date.