DMARC mitigation for mailing list server

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

DMARC mitigation for mailing list server

lucas2
Hi List,


I am running a mailing list server using the ListServ software. List
members can send a message to a list, and the software essentially
forwards the message to the entire list, using the following headers:
    Sender: <The local list mail address>
    From: <The mail address of the original sender of the message>
I use my own Postfix implementation as SMTP server to send the forwarded
message.

DMARC is increasingly causing problems for my list users, because the
"From:" sender does not match the sending server (which is my own
server).

One way to mitigate this problem would be to use the list address in
"From:". But the ListServ software does not support this.

My question: Is it possible to configure Postfix to replace the address
in the "From:" header with the value in the "Sender:" header?

If possible, the replacement should preferably be done for specific
values in the "Sender:" header, so it will not be implemented for all my
lists at the same time.

Kind regards,

Lucas
Reply | Threaded
Open this post in threaded view
|

Re: DMARC mitigation for mailing list server

Ralph Seichter-2
* lucas2:

> Is it possible to configure Postfix to replace the address in the
> "From:" header with the value in the "Sender:" header?

Modifying the "From" header is pretty much guaranteed to break existing
DKIM signatures (I have never seen anybody not sign "From"), so I doubt
that would mitigate your DMARC issues much.

-Ralph
Reply | Threaded
Open this post in threaded view
|

Re: DMARC mitigation for mailing list server

Matus UHLAR - fantomas
>* lucas2:
>> Is it possible to configure Postfix to replace the address in the
>> "From:" header with the value in the "Sender:" header?

On 26.03.19 17:50, Ralph Seichter wrote:
>Modifying the "From" header is pretty much guaranteed to break existing
>DKIM signatures (I have never seen anybody not sign "From"), so I doubt
>that would mitigate your DMARC issues much.

many mailing lists modify the "From:" header in order to create their own
DKIM signature pass and conform to DMARC.

However they don't replace it with contents of "Sender:" header but they
generate new one.

DMARC required the From: to be aligned with SPF mailfrom.

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
The early bird may get the worm, but the second mouse gets the cheese.
Reply | Threaded
Open this post in threaded view
|

Re: DMARC mitigation for mailing list server

Ralph Seichter-2
* Matus UHLAR:

>>Modifying the "From" header is pretty much guaranteed to break existing
>>DKIM signatures [...]
>
> many mailing lists modify the "From:" header in order to create their
> own DKIM signature pass and conform to DMARC.

Hence I wrote "break existing DKIM signatures".

-Ralph
Reply | Threaded
Open this post in threaded view
|

Re: DMARC mitigation for mailing list server

Bill Cole-3
On 26 Mar 2019, at 13:09, Ralph Seichter wrote:

> * Matus UHLAR:
>
>>> Modifying the "From" header is pretty much guaranteed to break
>>> existing
>>> DKIM signatures [...]
>>
>> many mailing lists modify the "From:" header in order to create their
>> own DKIM signature pass and conform to DMARC.
>
> Hence I wrote "break existing DKIM signatures".

Which is not a bad thing, in this context.

The problem is that most mailing lists routinely break DKIM signatures
anyway. When they do so without changing the From header, senders in
domains with a policy (p=) value in their DMARC record other than "none"
are at high risk of having their list postings rejected or quarantined
by sites honoring DMARC policies. If the address in the From header does
not align with the domain value in the DKIM-Signature header, the DMARC
policy of the signing domain is irrelevant.


--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Reply | Threaded
Open this post in threaded view
|

Re: DMARC mitigation for mailing list server

Ralph Seichter-2
* Bill Cole:

> > Hence I wrote "break existing DKIM signatures".
>
> Which is not a bad thing, in this context.

The OP made no mention of implementing DMARC himself, just modifying
headers. In that scenario, I consider breaking existing signatures a bad
thing. I am aware of alignment mechanics, but I see that tools like
SpamAssassin or Rspamd score signature (mis)matches individually, not
only in the context of DMARC policies.

As far as I can tell, modifying headers alone does not resolve the OP's
issues. By the way, I have recently started evaluating Mailman 3, which
comes with some interesting features re DMARC:

https://mailman.readthedocs.io/en/latest/src/mailman/rules/docs/dmarc-mitigation.html

-Ralph
Reply | Threaded
Open this post in threaded view
|

Re: DMARC mitigation for mailing list server

Wietse Venema
In reply to this post by lucas2
[hidden email]:

> Hi List,
>
>
> I am running a mailing list server using?the ListServ software. List
> members can send a message to a list, and the software essentially
> forwards the message to the entire list, using the?following headers:
> ? ? Sender: <The local list mail address>
> ? ? From: <The mail address of the original sender of the message>
> I use my own Postfix implementation as SMTP server to send the forwarded
> message.
>
> DMARC is increasingly causing problems for my list users, because the
> "From:" sender does not match the sending server (which is my own
> server).
>
> One way to mitigate this problem would be to use the list address in
> "From:". But the ListServ software does not support this.
>
> My question: Is it possible to configure?Postfix to replace the address
> in the "From:" header with the value in the "Sender:" header?
>
> If possible, the replacement should preferably be done for specific
> values in the "Sender:" header, so it will not be implemented for all my
> lists at the same time.

This would require a Milter or other content filter. Milters are
available in Perl, Python, and other languages. If someome could
write this up then I could add a note to the Postfix documentation.

For the Postfix side, see http://www.postfix.org/MILTER_README.html

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: DMARC mitigation for mailing list server

Miles Fidelman

> [hidden email]:
>> Hi List,
>>
>>
>> I am running a mailing list server using?the ListServ software. List
>> members can send a message to a list, and the software essentially
>> forwards the message to the entire list, using the?following headers:
>> ? ? Sender: <The local list mail address>
>> ? ? From: <The mail address of the original sender of the message>
>> I use my own Postfix implementation as SMTP server to send the forwarded
>> message.
>>
>> DMARC is increasingly causing problems for my list users, because the
>> "From:" sender does not match the sending server (which is my own
>> server).
>>
>> One way to mitigate this problem would be to use the list address in
>> "From:". But the ListServ software does not support this.
>>

L-Soft claims that ListServ is DMARC compatable - able to rewrite
headers to From: <the list> (same as you can do with Sympa & Mailman - I
believe Sympa had the first patch).  You might want to look at
https://www.lsoft.com/news/dmarc-issue1-2018.asp

Perhaps you need to play with your Listserv configuration a bit. Perhaps
you need an updated version.

It really isn't a Postfix issue at all.

Miles Fidelman (happily running Sympa, survived DMARC, so far)

--
In theory, there is no difference between theory and practice.
In practice, there is.  .... Yogi Berra

Theory is when you know everything but nothing works.
Practice is when everything works but no one knows why.
In our lab, theory and practice are combined:
nothing works and no one knows why.  ... unknown

Reply | Threaded
Open this post in threaded view
|

Re: DMARC mitigation for mailing list server

Matus UHLAR - fantomas
In reply to this post by Bill Cole-3
>>* Matus UHLAR:
>>
>>>>Modifying the "From" header is pretty much guaranteed to break
>>>>existing
>>>>DKIM signatures [...]
>>>
>>>many mailing lists modify the "From:" header in order to create their
>>>own DKIM signature pass and conform to DMARC.

>On 26 Mar 2019, at 13:09, Ralph Seichter wrote:
>>Hence I wrote "break existing DKIM signatures".

On 26.03.19 13:22, Bill Cole wrote:
>Which is not a bad thing, in this context.
>
>The problem is that most mailing lists routinely break DKIM signatures
>anyway.

usually when they prepend Subject with a text (e.g. list id).
Often they don't break DKIM.

> When they do so without changing the From header, senders in
>domains with a policy (p=) value in their DMARC record other than
>"none" are at high risk of having their list postings rejected or
>quarantined by sites honoring DMARC policies. If the address in the
>From header does not align with the domain value in the DKIM-Signature
>header, the DMARC policy of the signing domain is irrelevant.

if the mailing list doesn't modify existing headers, DKIM signatures are
valid but they don't align, so DMARC policy is violated.

DMARC sucks pretty much.

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
I intend to live forever - so far so good.
Reply | Threaded
Open this post in threaded view
|

Re: DMARC mitigation for mailing list server

Bill Cole-3
In reply to this post by Ralph Seichter-2
On 26 Mar 2019, at 13:39, Ralph Seichter wrote:

> * Bill Cole:
>
>>> Hence I wrote "break existing DKIM signatures".
>>
>> Which is not a bad thing, in this context.
>
> The OP made no mention of implementing DMARC himself, just modifying
> headers.

It's not about whether the list operator implements DMARC or DKIM.
Consider list members Alice and Bob:

Alice's domain has a p=reject DMARC policy.
Bob's mail provider honors p=reject DMARC policies.

Without From munging:
   Alice sends a message to the list which gets signed by her domain on
the way out and passed to the list operator.
   The list operator does something to the message that breaks the DKIM
signature.
   The list server tries to deliver the message to Bob, whose provider
rejects the message due to Alice's domain in the From header.

With From munging:
   Alice sends a message to the list which gets signed by her domain on
the way out and passed to the list operator.
   The list operator does something to the message that breaks the DKIM
signature.
   The list operator replaces the address in the From header with the
list submission address.
   The list server tries to deliver the message to Bob, whose provider
accepts the message.

One solution would be to not break DKIM signatures. However, this is
harder than it seems. For example, I see 24 recent DKIM-signed messages
from you to 3 different mailing lists that we both use. 6 have broken
signatures, all of those on 2 lists where not all of your messages have
broken signatures. I have no idea why the signatures broke.

> In that scenario, I consider breaking existing signatures a bad
> thing. I am aware of alignment mechanics, but I see that tools like
> SpamAssassin or Rspamd score signature (mis)matches individually, not
> only in the context of DMARC policies.

True. The convenience of having content scanners validate an aligned
signature has value. Unfortunately, the cost of NOT munging From for
list operators is either a large number of rejections at the outbound
border OR a constant battle with the DKIM-breaking edge and corner cases
of their particular mail-handling stack and its various configurations.

> As far as I can tell, modifying headers alone does not resolve the
> OP's
> issues. By the way, I have recently started evaluating Mailman 3,
> which
> comes with some interesting features re DMARC:
>
> https://mailman.readthedocs.io/en/latest/src/mailman/rules/docs/dmarc-mitigation.html

Mailman 2.1.x has similar features. You'll note that the mitigations
available are: discard, reject, munge the From header, or embed the
signed message with its pristine headers in a new message using a munged
 From header on the wrapper. None of these preserve an existing DKIM
signature in a generally useful form on a delivered message.

It would have been nice if the DKIM spec had defined the 'relaxed'
canonicalizations for headers and bodies more robustly but that can't be
fixed now.

--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Reply | Threaded
Open this post in threaded view
|

Re: DMARC mitigation for mailing list server

Bill Cole-3
In reply to this post by Matus UHLAR - fantomas
On 26 Mar 2019, at 14:47, Matus UHLAR - fantomas wrote:

> On 26.03.19 13:22, Bill Cole wrote:
> Which is not a bad thing, in this context.
>
> The problem is that most mailing lists routinely break DKIM signatures
> anyway.
>
> usually when they prepend Subject with a text (e.g. list id).
> Often they don't break DKIM.

Sometimes it is a mysterious Something Else. For an unknown reason, some
messages to this list get broken.


> if the mailing list doesn't modify existing headers, DKIM signatures
> are
> valid but they don't align, so DMARC policy is violated.

No: without modification of From, the original DKIM signature does align
with From, which is good enough that DMARC can pass IF the signature is
valid.



--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Reply | Threaded
Open this post in threaded view
|

Re: DMARC mitigation for mailing list server

Ralph Seichter-2
In reply to this post by Bill Cole-3
* Bill Cole:

> One solution would be to not break DKIM signatures. However, this is
> harder than it seems.

Not modifying messages' bodies or any signed headers seems to do the
trick. :-) With that in mind, I have recently filed an issue for Mailman
3, asking for configuration mechanics to disable all message decoration
(as screwing with the original is called there) on the levels of mailing
list, domain, and site.

As for doing my own part, I deliberately use a domain without DMARC
policy for mailing lists. Of course that won't prevent meddlesome
software from breaking my DKIM signatures, but at least nobody should
feel the urge to quarantine or reject my posts.

-Ralph
Reply | Threaded
Open this post in threaded view
|

Re: DMARC mitigation for mailing list server

Bill Cole-3
On 26 Mar 2019, at 15:41, Ralph Seichter wrote:

> * Bill Cole:
>
>> One solution would be to not break DKIM signatures. However, this is
>> harder than it seems.
>
> Not modifying messages' bodies or any signed headers seems to do the
> trick. :-)

Easier said than done, apparently.

About 5% of signed messages on this list are broken by the time they hit
my MX. That's a level which makes me feel pretty sure that something in
the postfix-users pipeline is making an otherwise harmless change to
those messages. Many use "simple" body canonicalization, which makes the
signature fragile. Others sign the Sender header, which the list server
legitimately adds to messages.


--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Reply | Threaded
Open this post in threaded view
|

Re: DMARC mitigation for mailing list server

Ralph Seichter-2
* Bill Cole:

> That's a level which makes me feel pretty sure that something in the
> postfix-users pipeline is making an otherwise harmless change to those
> messages.

I have not checked every single message, but I just inspected a few of
my own posts to this mailing list, and the signatures seem OK. I'm not
saying there is no breakage, just that I did not notice it.

> Many use "simple" body canonicalization, which makes the signature
> fragile.

Microsoft products were/are known to screw with email content, but I
don't think that any of them are included in this list's software stack?
As for body canonicalization, I'll try relaxed instead of simple, but I
still object to software messing with my message bodies, even when it
comes to whitespaces.

-Ralph
Reply | Threaded
Open this post in threaded view
|

Re: DMARC mitigation for mailing list server

Matus UHLAR - fantomas
In reply to this post by Bill Cole-3
>On 26 Mar 2019, at 14:47, Matus UHLAR - fantomas wrote:
>>if the mailing list doesn't modify existing headers, DKIM signatures
>>are
>>valid but they don't align, so DMARC policy is violated.

On 26.03.19 15:40, Bill Cole wrote:
>No: without modification of From, the original DKIM signature does
>align with From, which is good enough that DMARC can pass IF the
>signature is valid.

From what I know, the header From: (DKIM) is supposed to be aligned with
envelope from (SPF), which is not applicable for lists that keep header
From: but use their own envelope from.
https://en.wikipedia.org/wiki/DMARC#Mailing_lists

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Quantum mechanics: The dreams stuff is made of.
Reply | Threaded
Open this post in threaded view
|

Re: DMARC mitigation for mailing list server

Andrey Repin-2
Greetings, Matus UHLAR - fantomas!

>>On 26 Mar 2019, at 14:47, Matus UHLAR - fantomas wrote:
>>>if the mailing list doesn't modify existing headers, DKIM signatures
>>>are
>>>valid but they don't align, so DMARC policy is violated.

> On 26.03.19 15:40, Bill Cole wrote:
>>No: without modification of From, the original DKIM signature does
>>align with From, which is good enough that DMARC can pass IF the
>>signature is valid.

> From what I know, the header From: (DKIM) is supposed to be aligned with
> envelope from (SPF), which is not applicable for lists that keep header
> From: but use their own envelope from.
> https://en.wikipedia.org/wiki/DMARC#Mailing_lists

The topmost Resent-From should match envelope-from in this case.


--
With best regards,
Andrey Repin
Wednesday, March 27, 2019 10:57:27

Sorry for my terrible english...

Reply | Threaded
Open this post in threaded view
|

Re: DMARC mitigation for mailing list server

Bill Cole-3
In reply to this post by Matus UHLAR - fantomas
On 27 Mar 2019, at 3:51, Matus UHLAR - fantomas wrote:

>> On 26 Mar 2019, at 14:47, Matus UHLAR - fantomas wrote:
>>> if the mailing list doesn't modify existing headers, DKIM signatures
>>> are
>>> valid but they don't align, so DMARC policy is violated.
>
> On 26.03.19 15:40, Bill Cole wrote:
>> No: without modification of From, the original DKIM signature does
>> align with From, which is good enough that DMARC can pass IF the
>> signature is valid.
>
> From what I know, the header From: (DKIM) is supposed to be aligned
> with
> envelope from (SPF), which is not applicable for lists that keep
> header
> From: but use their own envelope from.

That is a misunderstanding of DMARC alignment. See
https://tools.ietf.org/html/rfc7489#section-3.1

If the From domain has a DMARC record, then at least one of DKIM and/or
SPF must authenticate a domain aligned to the From domain. Mailing lists
break alignment to SPF by necessity, so SPF authentication is not
relevant to DMARC and mailing lists. If the original From domain is used
in a DKIM signature, the mailing list must either perfectly avoid
breaking the signature validity (which is harder than it seems) or
change the From header so that its domain no longer has a DMARC record.

> https://en.wikipedia.org/wiki/DMARC#Mailing_lists

Wikipedia is not a good reference for any technical standard. In this
case, that section it is at best misleading and (as I read it,) simply
wrong.


--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Reply | Threaded
Open this post in threaded view
|

Re: DMARC mitigation for mailing list server

Bill Cole-3
In reply to this post by Ralph Seichter-2
On 26 Mar 2019, at 20:16, Ralph Seichter wrote:

> * Bill Cole:
>
>> That's a level which makes me feel pretty sure that something in the
>> postfix-users pipeline is making an otherwise harmless change to
>> those
>> messages.
>
> I have not checked every single message, but I just inspected a few of
> my own posts to this mailing list, and the signatures seem OK. I'm not
> saying there is no breakage, just that I did not notice it.

Most recent bad signature:

   Subject: Re: Rspamd as milter and 'discard' action
   Date: Thu, 14 Mar 2019 21:08:33 +0100
   Message-ID: <[hidden email]>

I see no obvious reason for it to have been modified in transit that
would break the signature. Everything after that and most before from
you to this list validated on arrival.

FWIW, I have sunk many hours recently (large billable, thankfully) into
diagnosing DKIM signature breakages and have been convinced that the
standard canonicalizations are inadequate.

>> Many use "simple" body canonicalization, which makes the signature
>> fragile.
>
> Microsoft products were/are known to screw with email content, but I
> don't think that any of them are included in this list's software
> stack?

It appears not.
Sendmail can also do damage but I don't see it involved.
My bet would be on majordomo, if I were a betting man.

> As for body canonicalization, I'll try relaxed instead of simple, but
> I
> still object to software messing with my message bodies, even when it
> comes to whitespaces.

Yes, but 'harmless cleanup' is a widespread practice. I don't see much
chance of totally eradicating it, especially in mailing list software.

--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Reply | Threaded
Open this post in threaded view
|

Re: DMARC mitigation for mailing list server

Ralph Seichter-2
* Bill Cole:

> Most recent bad signature:
>
>   Subject: Re: Rspamd as milter and 'discard' action
>   Date: Thu, 14 Mar 2019 21:08:33 +0100
>   Message-ID: <[hidden email]>

Weird. I have just verified the raw message, using both 'dkimpy' and
http://www.appmaildev.com/en/dkimfile , and in both cases the signature
was reported as OK, same as it was originally reported by Rspamd.

-Ralph
Reply | Threaded
Open this post in threaded view
|

Re: DMARC mitigation for mailing list server

Bill Cole-3
On 28 Mar 2019, at 13:09, Ralph Seichter wrote:

> * Bill Cole:
>
>> Most recent bad signature:
>>
>>   Subject: Re: Rspamd as milter and 'discard' action
>>   Date: Thu, 14 Mar 2019 21:08:33 +0100
>>   Message-ID: <[hidden email]>
>
> Weird. I have just verified the raw message, using both 'dkimpy' and
> http://www.appmaildev.com/en/dkimfile , and in both cases the
> signature
> was reported as OK, same as it was originally reported by Rspamd.

Please accept my apology for wasting your time, and thank you for the
link to that validator.

I just put the message as it was delivered here into that page and DKIM
passed. Apparently I have a problem locally, possibly a bug in
SpamAssassin's DKIM validation.

--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
12