DANE issue with postfix 3.4.0-RC2

classic Classic list List threaded Threaded
32 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Patch: 3.4.0-RC2 and 3.5 snapshots (was: DANE issue with postfix 3.4.0-RC2)

Viktor Dukhovni
On Mon, Feb 18, 2019 at 02:48:32PM -0500, Wietse Venema wrote:

> > > Should we remove the those calls and make tls_pre_jail_init() a
> > > mandatory call?
> >
> > I considered making the pre-jail init mandatory, but decided not
> > to mess with posttls-finger, and left them in place.
>
> We should make tls_pre_jail_init() mandatory if that is the only
> way to guarantee that shit won't blow up.

Well, presently it is only the tlsproxy that looks at the TLS
parameters before calling tls_client_init().  But it does call the
pre-jail init first, and it does make sense to load all the parameters
there.  The likelihood of needing similar early access elsewhere
seems low, but for the record I actually had the mandatory pre-jail
init change done, before backing it out...  So it is not a wild idea.

> > The tls_dane_avail() function tests for and initializes optional
> > library features, that are pre-requisites for DANE, but not necessarily
> > for doing TLS generally.  It is not a policy decision, but it is
> > deferred until DANE is actually used for the first time.
>
> The Postfix SMTP client already decides if DANE is available. Why
> is this needed again in tlsproxy? I see this as a problem in the
> architecture if the tlsproxy client cannot simply delegate some TLS
> library calls to the tlsproxy server.

Modulo code upgrades without a restart, indeed the proxy and client
should have equally capable libraries.  So DANE could be "ready to
go" (when available) in the proxy from the outset, by always enabling
library support.  With OpenSSL 1.0.0 and 1.0.1, we would often find
incomplete support, and would not be able to report the reasons why
DANE is not supported, if initialization is unconditional even for
sites that don't enable outbound DANE>

With OpenSSL 1.0.2, all the pre-requisites are there, so with a bit
of refactoring, we could initialize the DANE TLS support unconditionally
in tls_client_init(), so it is ready to run.  We'd probably move
the call to tls_dane_avail() into tls_client_start(), and fail there
if the security level is DANE-based, but there's no DANE support.

The tls_dane_avail() call would no longer be responsible for
DANE initialization, it would just examine the memoized result.

Do you want to do that now in a 3.4.0-RC3?  Or save the cleanup
for 3.5?

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Patch: 3.4.0-RC2 and 3.5 snapshots (was: DANE issue with postfix 3.4.0-RC2)

Wietse Venema
Viktor Dukhovni:
> Do you want to do that now in a 3.4.0-RC3?  Or save the cleanup
> for 3.5?

I wanted to understand why the code is "organized" as it is, as
kinda sorta parallel worlds, instead of client-server style delegation.

I understand that with the proposed code organization, we can release
Postfix 3.4.0 responsibly (with some warts in the way we share SSL_CTX).

But I also want to get this straightened out in Postfix 3.5, back
to a simple remote call model that does support delegation, and
with only explicit state sharing so that tlsproxy can handle different
kinds of servers and clients.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Patch: 3.4.0-RC2 and 3.5 snapshots (was: DANE issue with postfix 3.4.0-RC2)

A. Schulze
In reply to this post by Viktor Dukhovni

Viktor Dukhovni:

> diff --git a/src/tls/tls_misc.c b/src/tls/tls_misc.c
> diff --git a/src/tlsproxy/tlsproxy.c b/src/tlsproxy/tlsproxy.c


> Another issue remains, in that tlsproxy(8) wants
> unconditional server-side support before it is willing to be a
> client proxy, and therefore also wants server certificates.

that 'wants server certificates' becomes reality on on of my lab systems :-)

$ tail /var/log/messages
Feb 19 13:25:53 spider kernel: [93723.068442] tlsproxy[996]: segfault  
at 0 ip 000055c62130f9b7 sp 00007ffc3e867f80 error 4 in  
tlsproxy[55c6212c4000+6a000]
Feb 19 13:26:53 spider kernel: [93783.109960] tlsproxy[1021]: segfault  
at 0 ip 00005581336789b7 sp 00007fffbc8b67d0 error 4 in  
tlsproxy[55813362d000+6a000]
Feb 19 13:27:53 spider kernel: [93843.173476] tlsproxy[1046]: segfault  
at 0 ip 00005606426ba9b7 sp 00007ffe5cd674a0 error 4 in  
tlsproxy[56064266f000+6a000]
Feb 19 13:28:53 spider kernel: [93903.201985] tlsproxy[1061]: segfault  
at 0 ip 0000556dff47c9b7 sp 00007ffceb0de480 error 4 in  
tlsproxy[556dff431000+6a000]

$ tail /var/log/mail
Feb 19 13:25:53 spider postfix/master[2282]: warning: process  
/usr/lib/postfix/tlsproxy pid 996 killed by signal 11
Feb 19 13:25:53 spider postfix/master[2282]: warning:  
/usr/lib/postfix/tlsproxy: bad command startup -- throttling
Feb 19 13:26:53 spider postfix/tlsproxy[1021]: warning: No server  
certs available. TLS can't be enabled
Feb 19 13:26:53 spider postfix/master[2282]: warning: process  
/usr/lib/postfix/tlsproxy pid 1021 killed by signal 11
Feb 19 13:26:53 spider postfix/master[2282]: warning:  
/usr/lib/postfix/tlsproxy: bad command startup -- throttling
Feb 19 13:27:53 spider postfix/tlsproxy[1046]: warning: No server  
certs available. TLS can't be enabled
Feb 19 13:27:53 spider postfix/master[2282]: warning: process  
/usr/lib/postfix/tlsproxy pid 1046 killed by signal 11
Feb 19 13:27:53 spider postfix/master[2282]: warning:  
/usr/lib/postfix/tlsproxy: bad command startup -- throttling
Feb 19 13:28:53 spider postfix/tlsproxy[1061]: warning: No server  
certs available. TLS can't be enabled
Feb 19 13:28:53 spider postfix/master[2282]: warning: process  
/usr/lib/postfix/tlsproxy pid 1061 killed by signal 11
Feb 19 13:28:53 spider postfix/master[2282]: warning:  
/usr/lib/postfix/tlsproxy: bad command startup -- throttling

The discussion between Wietse and Viktor addresses exactly that point.
I only want to show/agree, Viktors Patch isn't ready to release.

Andreas


Reply | Threaded
Open this post in threaded view
|

Re: Patch: 3.4.0-RC2 and 3.5 snapshots (was: DANE issue with postfix 3.4.0-RC2)

A. Schulze

A. Schulze:

> Viktor Dukhovni:
>
>> diff --git a/src/tls/tls_misc.c b/src/tls/tls_misc.c
>> diff --git a/src/tlsproxy/tlsproxy.c b/src/tlsproxy/tlsproxy.c

there is an other side effect:

I configured
smtpd_tls_cert_file = /etc/ssl/${myhostname}/cert+intermediate.pem
smtpd_tls_key_file = /etc/ssl/${myhostname}/key.pem
smtp_tls_cert_file = /etc/ssl/${myhostname}/cert+intermediate.pem
smtp_tls_key_file = /etc/ssl/${myhostname}/key.pem

now, on my lab maschine, a little bit complicated via
     sender_dependent_relayhost_maps
         @example.org [mail.example.org]:465

and to enforce the (local required) smtp_wrapper_mode
     sender_dependent_default_transport_maps
         @example.org submissions:

"submissions" is defined in master.cf:
     submissions                            unix  - - y -     - smtp
      -o smtp_tls_security_level=encrypt
      -o smtp_tls_wrappermode=yes
      -o syslog_name=postfix/${service_name}

now, "sendmail -f [hidden email] -bc [hidden email]" throw  
this error:

Feb 19 14:24:09 spider postfix/pickup[3865]: 443hK512TRzMvsx7:  
uid=1000 from=<[hidden email]>
Feb 19 14:24:09 spider postfix/cleanup[3869]: 443hK512TRzMvsx7:  
message-id=<443hK512TRzMvsx7@$myhostname>
Feb 19 14:24:09 spider postfix/qmgr[3866]: 443hK512TRzMvsx7:  
from=<sender@example>, size=302, nrcpt=1 (queue active)
Feb 19 14:24:09 spider postfix/tlsproxy[3873]: CONNECT to [192.0.2.25]:465
Feb 19 14:24:09 spider postfix/submissions/smtp[3895]: panic:  
VSTREAM_CTL_SWAP_FD can't swap descriptors between single-buffered and  
double-buffered streams
Feb 19 14:24:09 spider postfix/tlsproxy[3873]: Trusted TLS connection  
established to mail.example.org[192.0.2.25]:465: TLSv1.3 with cipher  
TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384)  
server-signature RSA-PSS (4096 bits) server-digest SHA256
Feb 19 14:24:10 spider postfix/qmgr[3866]: warning:  
private/submissions socket: malformed response
Feb 19 14:24:10 spider postfix/qmgr[3866]: warning: transport  
submissions failure -- see a previous warning/fatal/panic logfile  
record for the problem description
Feb 19 14:24:10 spider postfix/master[2282]: warning: process  
/usr/lib/postfix/smtp pid 3895 killed by signal 6
Feb 19 14:24:10 spider postfix/master[2282]: warning:  
/usr/lib/postfix/smtp: bad command startup -- throttling
Feb 19 14:24:10 spider postfix/tlsproxy[3873]: DISCONNECT [192.0.2.25]:465
Feb 19 14:24:10 spider postfix/error[3875]: 443hK512TRzMvsx7:  
to=<[hidden email]>, relay=none, delay=1, delays=0.02/1/0/0.01,  
dsn=4.3.0, status=undeliverable (unknown mail transport error)
( last line isn't the surprise ... )

I guess it's related to my previous posting.

Andreas



Reply | Threaded
Open this post in threaded view
|

Re: Patch: 3.4.0-RC2 and 3.5 snapshots (was: DANE issue with postfix 3.4.0-RC2)

Viktor Dukhovni
In reply to this post by A. Schulze
> On Feb 19, 2019, at 7:35 AM, A. Schulze <[hidden email]> wrote:
>
> Feb 19 13:25:53 spider postfix/master[2282]: warning: process /usr/lib/postfix/tlsproxy pid 996 killed by signal 11
> Feb 19 13:25:53 spider postfix/master[2282]: warning: /usr/lib/postfix/tlsproxy: bad command startup -- throttling
> Feb 19 13:26:53 spider postfix/tlsproxy[1021]: warning: No server certs available. TLS can't be enabled
> Feb 19 13:26:53 spider postfix/master[2282]: warning: process /usr/lib/postfix/tlsproxy pid 1021 killed by signal 11
> Feb 19 13:26:53 spider postfix/master[2282]: warning: /usr/lib/postfix/tlsproxy: bad command startup -- throttling
> Feb 19 13:27:53 spider postfix/tlsproxy[1046]: warning: No server certs available. TLS can't be enabled
> Feb 19 13:27:53 spider postfix/master[2282]: warning: process /usr/lib/postfix/tlsproxy pid 1046 killed by signal 11
> Feb 19 13:27:53 spider postfix/master[2282]: warning: /usr/lib/postfix/tlsproxy: bad command startup -- throttling
> Feb 19 13:28:53 spider postfix/tlsproxy[1061]: warning: No server certs available. TLS can't be enabled
> Feb 19 13:28:53 spider postfix/master[2282]: warning: process /usr/lib/postfix/tlsproxy pid 1061 killed by signal 11
> Feb 19 13:28:53 spider postfix/master[2282]: warning: /usr/lib/postfix/tlsproxy: bad command startup -- throttling
>
> The discussion between Wietse and Viktor addresses exactly that point.
> I only want to show/agree, Viktors Patch isn't ready to release.

Is that *with* my patch applied?  For me, applying the patch
made the segfaults in a certificateless proxy configuration
go away.  I see the same segfaults with 3.4.0-RC2 unpatched,
but not with the patch...

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: Patch: 3.4.0-RC2 and 3.5 snapshots (was: DANE issue with postfix 3.4.0-RC2)

A. Schulze


Am 19.02.19 um 15:37 schrieb Viktor Dukhovni:

>> On Feb 19, 2019, at 7:35 AM, A. Schulze <[hidden email]> wrote:
>>
>> Feb 19 13:25:53 spider postfix/master[2282]: warning: process /usr/lib/postfix/tlsproxy pid 996 killed by signal 11
>> Feb 19 13:25:53 spider postfix/master[2282]: warning: /usr/lib/postfix/tlsproxy: bad command startup -- throttling
>> Feb 19 13:26:53 spider postfix/tlsproxy[1021]: warning: No server certs available. TLS can't be enabled
>> Feb 19 13:26:53 spider postfix/master[2282]: warning: process /usr/lib/postfix/tlsproxy pid 1021 killed by signal 11
>> Feb 19 13:26:53 spider postfix/master[2282]: warning: /usr/lib/postfix/tlsproxy: bad command startup -- throttling
>> Feb 19 13:27:53 spider postfix/tlsproxy[1046]: warning: No server certs available. TLS can't be enabled
>> Feb 19 13:27:53 spider postfix/master[2282]: warning: process /usr/lib/postfix/tlsproxy pid 1046 killed by signal 11
>> Feb 19 13:27:53 spider postfix/master[2282]: warning: /usr/lib/postfix/tlsproxy: bad command startup -- throttling
>> Feb 19 13:28:53 spider postfix/tlsproxy[1061]: warning: No server certs available. TLS can't be enabled
>> Feb 19 13:28:53 spider postfix/master[2282]: warning: process /usr/lib/postfix/tlsproxy pid 1061 killed by signal 11
>> Feb 19 13:28:53 spider postfix/master[2282]: warning: /usr/lib/postfix/tlsproxy: bad command startup -- throttling
>>
>> The discussion between Wietse and Viktor addresses exactly that point.
>> I only want to show/agree, Viktors Patch isn't ready to release.
>
> Is that *with* my patch applied?  For me, applying the patch
> made the segfaults in a certificateless proxy configuration
> go away.  I see the same segfaults with 3.4.0-RC2 unpatched,
> but not with the patch...

indeed, my fault / I falsely assumed the autoupdater did it's job.
The tlsproxy start without noise in certificateless configuration.

Sorry for not checking that earlier.

Andreas
Reply | Threaded
Open this post in threaded view
|

Re: Patch: 3.4.0-RC2 and 3.5 snapshots (was: DANE issue with postfix 3.4.0-RC2)

A. Schulze
In reply to this post by A. Schulze


Am 19.02.19 um 14:28 schrieb A. Schulze:

>
> A. Schulze:
>
>> Viktor Dukhovni:
>>
>>> diff --git a/src/tls/tls_misc.c b/src/tls/tls_misc.c
>>> diff --git a/src/tlsproxy/tlsproxy.c b/src/tlsproxy/tlsproxy.c
>
> there is an other side effect:
>
> I configured
> smtpd_tls_cert_file = /etc/ssl/${myhostname}/cert+intermediate.pem
> smtpd_tls_key_file = /etc/ssl/${myhostname}/key.pem
> smtp_tls_cert_file = /etc/ssl/${myhostname}/cert+intermediate.pem
> smtp_tls_key_file = /etc/ssl/${myhostname}/key.pem
>
> now, on my lab maschine, a little bit complicated via
>     sender_dependent_relayhost_maps
>         @example.org [mail.example.org]:465
>
> and to enforce the (local required) smtp_wrapper_mode
>     sender_dependent_default_transport_maps
>         @example.org submissions:
>
> "submissions" is defined in master.cf:
>     submissions                            unix  - - y -     - smtp
>      -o smtp_tls_security_level=encrypt
>      -o smtp_tls_wrappermode=yes
>      -o syslog_name=postfix/${service_name}
>
> now, "sendmail -f [hidden email] -bc [hidden email]" throw this error:
>
> Feb 19 14:24:09 spider postfix/pickup[3865]: 443hK512TRzMvsx7: uid=1000 from=<[hidden email]>
> Feb 19 14:24:09 spider postfix/cleanup[3869]: 443hK512TRzMvsx7: message-id=<443hK512TRzMvsx7@$myhostname>
> Feb 19 14:24:09 spider postfix/qmgr[3866]: 443hK512TRzMvsx7: from=<sender@example>, size=302, nrcpt=1 (queue active)
> Feb 19 14:24:09 spider postfix/tlsproxy[3873]: CONNECT to [192.0.2.25]:465
> Feb 19 14:24:09 spider postfix/submissions/smtp[3895]: panic: VSTREAM_CTL_SWAP_FD can't swap descriptors between single-buffered and double-buffered streams
> Feb 19 14:24:09 spider postfix/tlsproxy[3873]: Trusted TLS connection established to mail.example.org[192.0.2.25]:465: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256
> Feb 19 14:24:10 spider postfix/qmgr[3866]: warning: private/submissions socket: malformed response
> Feb 19 14:24:10 spider postfix/qmgr[3866]: warning: transport submissions failure -- see a previous warning/fatal/panic logfile record for the problem description
> Feb 19 14:24:10 spider postfix/master[2282]: warning: process /usr/lib/postfix/smtp pid 3895 killed by signal 6
> Feb 19 14:24:10 spider postfix/master[2282]: warning: /usr/lib/postfix/smtp: bad command startup -- throttling
> Feb 19 14:24:10 spider postfix/tlsproxy[3873]: DISCONNECT [192.0.2.25]:465
> Feb 19 14:24:10 spider postfix/error[3875]: 443hK512TRzMvsx7: to=<[hidden email]>, relay=none, delay=1, delays=0.02/1/0/0.01, dsn=4.3.0, status=undeliverable (unknown mail transport error)
> ( last line isn't the surprise ... )
>
> I guess it's related to my previous posting.
>
> Andreas
>

but this error stay even with Viktor's patch applied.
smtpd_tls_cert_file + smtpd_tls_key_file need to be set, smtp_tls_cert_file+smtp_tls_key_file don't matter.


Andreas
Reply | Threaded
Open this post in threaded view
|

Re: Patch: 3.4.0-RC2 and 3.5 snapshots (was: DANE issue with postfix 3.4.0-RC2)

Viktor Dukhovni
In reply to this post by A. Schulze
> On Feb 19, 2019, at 10:50 AM, A. Schulze <[hidden email]> wrote:
>
>>> Feb 19 13:28:53 spider postfix/tlsproxy[1061]: warning: No server certs available. TLS can't be enabled
>>
>> For me, applying the patch
>> made the segfaults in a certificateless proxy configuration
>> go away.
>
> indeed, my fault / I falsely assumed the autoupdater did it's job.
> The tlsproxy start without noise in certificateless configuration.

Thanks.  Mind you I still see the "TLS can't be enabled message" logged
at proxy startup, but what it means is that *server-side* TLS is not
available, the client side still works.  I've not tested what happens
with server-side operation in that case, does it refuse service, or
try and fail in some manner?

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: Patch: 3.4.0-RC2 and 3.5 snapshots (was: DANE issue with postfix 3.4.0-RC2)

Viktor Dukhovni
> On Feb 19, 2019, at 11:15 AM, A. Schulze <[hidden email]> wrote:
>
>> I've not tested what happens
>> with server-side operation in that case, does it refuse service, or
>> try and fail in some manner?
>
> unsure if that's a open question or you expect me something to test?

I am not sure either. :-)  I you want to help out, you could test
postscreen's use of the server-side proxy with deep-protocol
tests over TLS.  I don't know how the proxy behaves when the server
initialization does not run for lack of certificates.

You'd still need to convince postscreen that it should offer TLS...

I don't recall whether use of tlsproxy(8) with smtpd(8) is supported,
there's presently no good reason to do that.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

PATCH: tls reuse and wrappermode (port 465)

Wietse Venema
In reply to this post by A. Schulze
A. Schulze:
> > Feb 19 14:24:09 spider postfix/submissions/smtp[3895]: panic: VSTREAM_CTL_SWAP_FD can't swap descriptors between single-buffered and double-buffered streams

That was the result of wrappermode never having been tested with
tlsproxy mode turned on.

        Wietse

*** src/smtp_proto.c- 2019-02-08 17:22:24.000000000 -0500
--- src/smtp_proto.c 2019-02-19 13:46:58.000000000 -0500
***************
*** 337,342 ****
--- 337,344 ----
  && (state->misc_flags & SMTP_MISC_FLAG_IN_STARTTLS) == 0) {
  /* XXX Mix-up of per-session and per-request flags. */
  state->misc_flags |= SMTP_MISC_FLAG_IN_STARTTLS;
+ smtp_stream_setup(state->session->stream, var_smtp_helo_tmout,
+  var_smtp_rec_deadline);
  tls_helo_status = smtp_start_tls(state);
  state->misc_flags &= ~SMTP_MISC_FLAG_IN_STARTTLS;
  return (tls_helo_status);
Reply | Threaded
Open this post in threaded view
|

Re: PATCH: tls reuse and wrappermode (port 465)

A. Schulze


Am 19.02.19 um 19:48 schrieb Wietse Venema:

> A. Schulze:
>>> Feb 19 14:24:09 spider postfix/submissions/smtp[3895]: panic: VSTREAM_CTL_SWAP_FD can't swap descriptors between single-buffered and double-buffered streams
>
> That was the result of wrappermode never having been tested with
> tlsproxy mode turned on.
>
> Wietse
>
> *** src/smtp_proto.c- 2019-02-08 17:22:24.000000000 -0500
> --- src/smtp_proto.c 2019-02-19 13:46:58.000000000 -0500
> ***************
> *** 337,342 ****
> --- 337,344 ----
>   && (state->misc_flags & SMTP_MISC_FLAG_IN_STARTTLS) == 0) {
>   /* XXX Mix-up of per-session and per-request flags. */
>   state->misc_flags |= SMTP_MISC_FLAG_IN_STARTTLS;
> + smtp_stream_setup(state->session->stream, var_smtp_helo_tmout,
> +  var_smtp_rec_deadline);
>   tls_helo_status = smtp_start_tls(state);
>   state->misc_flags &= ~SMTP_MISC_FLAG_IN_STARTTLS;
>   return (tls_helo_status);
>

Hello Wietse,

that fixes the error...

Andreas
Reply | Threaded
Open this post in threaded view
|

Re: PATCH: tls reuse and wrappermode (port 465)

Wietse Venema
A. Schulze:

>
>
> Am 19.02.19 um 19:48 schrieb Wietse Venema:
> > A. Schulze:
> >>> Feb 19 14:24:09 spider postfix/submissions/smtp[3895]: panic: VSTREAM_CTL_SWAP_FD can't swap descriptors between single-buffered and double-buffered streams
> >
> > That was the result of wrappermode never having been tested with
> > tlsproxy mode turned on.
> >
> > Wietse
> >
> > *** src/smtp_proto.c- 2019-02-08 17:22:24.000000000 -0500
> > --- src/smtp_proto.c 2019-02-19 13:46:58.000000000 -0500
> > ***************
> > *** 337,342 ****
> > --- 337,344 ----
> >   && (state->misc_flags & SMTP_MISC_FLAG_IN_STARTTLS) == 0) {
> >   /* XXX Mix-up of per-session and per-request flags. */
> >   state->misc_flags |= SMTP_MISC_FLAG_IN_STARTTLS;
> > + smtp_stream_setup(state->session->stream, var_smtp_helo_tmout,
> > +  var_smtp_rec_deadline);
> >   tls_helo_status = smtp_start_tls(state);
> >   state->misc_flags &= ~SMTP_MISC_FLAG_IN_STARTTLS;
> >   return (tls_helo_status);
> >
>
> Hello Wietse,
>
> that fixes the error...

Good. I will roll out 3.5 snapshots, but no 3.4.0 candidate until there
are no more errors.

        Wietse
12