I'm using Postfix's XCLIENT to synthesize/inject a test email into my postfix->filter/milter->delivery chain.
I'd like to verify that my XCLIENT usage isn't the cause of the delivery failure I see below ... @ this postfix instance, mail flows as -> postscreen (@ IP = 203.0.113.1) | internal smptd | spf policy engine (no reject; check + header only) | preQ milters: opendkim, opendmarc, clamav-milter, spamassassin-milter | lmtp -> dovecot I've no inbound/outbound issues. Except ... ... from a single source -- @intuit.com. it's FAIL'ing @ opendmarc checks. I've online-checked SPF/DMARC records for 'intuit.com'; all _seems_ to be ok. I've cranked up opendmarc logging level to MilterDebug 5 with that, on failed attempt, I see only an unhelpful Oct 21 09:43:39 mx.example.com opendmarc[7977]: 4CGbb3aX1Pz2N: intuit.com fail I'm trying to use XCLIENT to replicate the issue so I can test, rinse & repeat. Trying 1st from @gmail.com (or any domain i've tried _other_ than 'intuit.com') using data pulled from postfix logs for a SUCCESSFUL [hidden email] delivery, @ an opened 'openssl s_client' session to my postfix external IP, injecting XCLIENT NAME=mail-vs1-f46.google.com ADDR=209.85.217.46 PORT=40169 PROTO=ESMTP HELO=mail-vs1-f46.google.com DESTADDR=203.0.113.1 DESTPORT=25 MAIL FROM:<[hidden email]> RCPT TO:<[hidden email]> DATA test message (CR/LF) . (CR/LF) mail passes all filters, and is delivered cleanly. I'm able to repeat the success for any/all of the non-intuit.com senders I try. Switching to the data pulled from postfix logs for a FAILED [hidden email] delivery, again @ an opened 'openssl s_client' session to my postfix external IP, injecting XCLIENT NAME=55.57.138.139.in-addr.arpa.iphmx.com ADDR=139.138.57.55 PORT=62440 PROTO=ESMTP HELO=esa3.hc3812-35.iphmx.com DESTADDR=203.0.113.1 DESTPORT=25 MAIL FROM:<[hidden email]> RCPT TO:<[hidden email]> DATA test message (CR/LF) . (CR/LF) fails in the session with 550 5.7.1 rejected by DMARC policy for intuit.com and is not delivered. Before I take this up as an opendmarc question (my config &/or bug), & do more thorough digging re: intuit's published records, (1) Is there anything obviously wrong/missing in that^ XCLIENT usage generally, or in the specific intuit.com case above, that would suggest a cause for the dmarc/milter FAIL, that 1st needs fixing? I _suspect_ not, given the success with all _other_ domains ... |
If your XCLIENT arguments match Postfix logging, including the name
and IP address info and you used HELO or EHLO depending on Postfix's proto= logging, then I think that the Postfix SMTP daemon cannot distinguish between a real intuit.com connection and one made with XCLIENT. That leaves the possibility of - Some Postfix configuration affects intuit.com mail but not other mail. For example a false match with an inaccurate regular expression! - Copy and paste problem with message header or body content - Local DMARC etc. configuration problem - Sender's DNARC etc. configuration problem The DMARC logging shouod give an idea of what the problem is. Wietse |
On 10/21/20 11:13 AM, Wietse Venema wrote:
> If your XCLIENT arguments match Postfix logging, including the name and IP address info they do > and you used HELO or EHLO depending on Postfix's proto= logging proto=ESMTP, so I used EHLO > then I think that the Postfix SMTP daemon cannot distinguish between a real intuit.com connection and one made with XCLIENT. useful to know. so likely downstream, NOT xclient. > That leaves the possibility of > > - Some Postfix configuration affects intuit.com mail but not other mail. > For example a false match with an inaccurate regular expression! nothing intuit-specific, and also ripped out all regex matching to test. same > - Copy and paste problem with message header or body content double checked. same > - Sender's DMARC etc. configuration problem If so, online checks I've used aren't finding it. I understand they're not necessarily exhaustive. > - Local DMARC etc. configuration problem > > The DMARC logging shouod give an idea of what the problem is. I still haven't managed to get opendmarc to cough up more than the reported unhelpful one-liner in logs. Which, in addition to the lack of maintenance of the software, makes me increasingly suspicious. I can do some debug with my own builds; see how far that gets me. Two questions: (1) my postfix config includes, strict_rfc821_envelopes = yes the FROM: & RCPT TO: addressed i inject, as well as those in the originally sent mail, appear to be compliant. is there _more_ that strict restriction that might be relevant? (2) currently, the opendmarc milter here is launched as a 'simple' non-forking/foreground systemd service [Service] Type=simple ExecStart=/usr/sbin/opendmarc \ -f \ -c /etc/opendmarc/opendmarc.conf \ -u opendmarc been that way for ages. and obviously works, generally. checking newer implementations online, i've switched to a forking service, [Service] Type=forking ExecStart=/usr/sbin/opendmarc \ -c /etc/opendmarc/opendmarc.conf \ -u opendmarc Initially, I see no difference. At least not with the XCLIENT injection tests. For milter use, in postfix, is there any preference/requirement for the milters to exec in forked vs. un-forked mode? |
PGNet Dev:
> Two questions: > > (1) my postfix config includes, > > strict_rfc821_envelopes = yes > > the FROM: & RCPT TO: addressed i inject, as well as those in the originally sent mail, appear to be compliant. > > is there _more_ that strict restriction that might be relevant? Postfix would have logged any hostile actions.. > (2) currently, the opendmarc milter here is launched as a 'simple' non-forking/foreground systemd service I don't expect that the forking model would make a difference for how DMARC etc. policy is evaluated. > Initially, I see no difference. At least not with the XCLIENT > injection tests. > > For milter use, in postfix, is there any preference/requirement > for the milters to exec in forked vs. un-forked mode? Postfix does not care. If forked versus non-forked would change the bits that the milter receives or sends, then that would be a defect on the milter side (it can happen when they fork in the middle of a postfix-milter conversation that is already in progress, but the cole is old enough that it would be unlikely). Wietse |
On 10/21/20 4:31 PM, Wietse Venema wrote:
> PGNet Dev: >> Two questions: clear. i'll focus just on just the dmarc bits. both debugging opendmarc, and replacing it with another option to see if behavior changes. xclient's extremely helpful in any case. |
On 22/10/2020 00:39, PGNet Dev wrote:
> On 10/21/20 4:31 PM, Wietse Venema wrote: >> PGNet Dev: >>> Two questions: > > clear. > > i'll focus just on just the dmarc bits. > > both debugging opendmarc, and replacing it with another option to see > if behavior changes. > > xclient's extremely helpful in any case. It may be unrelated, but we have received a few fake intuit emails recently, all correctly identified as such by opendmarc. |
In reply to this post by PGNet Dev
Hi name less
On Wed, Oct 21, 2020 at 10:13:54AM -0700, PGNet Dev wrote: > I've online-checked SPF/DMARC records for 'intuit.com'; all _seems_ to be ok. > I've cranked up opendmarc logging level to > MilterDebug 5 > with that, on failed attempt, I see only an unhelpful > Oct 21 09:43:39 mx.example.com opendmarc[7977]: 4CGbb3aX1Pz2N: intuit.com fail This is not Postfix! > Trying 1st from @gmail.com (or any domain i've tried _other_ than 'intuit.com') Please see the DMARC policy of gmail.com, especially the "none" policy: | _dmarc.gmail.com. IN TXT "v=DMARC1; p=none; sp=quarantine; […]" > using data pulled from postfix logs for a SUCCESSFUL [hidden email] delivery, > @ an opened 'openssl s_client' session to my postfix external IP, injecting > > XCLIENT NAME=mail-vs1-f46.google.com ADDR=209.85.217.46 PORT=40169 PROTO=ESMTP HELO=mail-vs1-f46.google.com DESTADDR=203.0.113.1 DESTPORT=25 > MAIL FROM:<[hidden email]> > RCPT TO:<[hidden email]> > DATA > test message > (CR/LF) > . > (CR/LF) This mail is not signed by gmail.com! But as the policy is none, it's not rejected or otherwise handled. > Switching to the data pulled from postfix logs for a FAILED [hidden email] delivery, > again @ an opened 'openssl s_client' session to my postfix external IP, injecting Please see the DMARC policy of intuit.com, especially the "reject" policy: | _dmarc.intuit.com. IN TXT "v=DMARC1; p=reject; […]" > XCLIENT NAME=55.57.138.139.in-addr.arpa.iphmx.com ADDR=139.138.57.55 PORT=62440 PROTO=ESMTP HELO=esa3.hc3812-35.iphmx.com DESTADDR=203.0.113.1 DESTPORT=25 > MAIL FROM:<[hidden email]> > RCPT TO:<[hidden email]> > DATA > test message > (CR/LF) > . > (CR/LF) > > fails in the session with > > 550 5.7.1 rejected by DMARC policy for intuit.com > > and is not delivered. This mail is not signed by intuit.com! And SPF interaction is pretty weird for the HELO-only case. So you _must_ use a real signed e-mail to check DMARC interaction. > (1) Is there anything obviously wrong/missing in that^ XCLIENT usage generally, or in the specific intuit.com case above, that would suggest a cause for the dmarc/milter FAIL, that 1st needs fixing? No, but it changes almost nothing for DMARC interactions, as DMARC looks on header information, not envelope in almost all cases. Bastian -- Virtue is a relative term. -- Spock, "Friday's Child", stardate 3499.1 |
On 10/22/20 3:23 AM, Bastian Blank wrote:
> Hi name less > > On Wed, Oct 21, 2020 at 10:13:54AM -0700, PGNet Dev wrote: >> I've online-checked SPF/DMARC records for 'intuit.com'; all _seems_ to be ok. >> I've cranked up opendmarc logging level to >> MilterDebug 5 >> with that, on failed attempt, I see only an unhelpful >> Oct 21 09:43:39 mx.example.com opendmarc[7977]: 4CGbb3aX1Pz2N: intuit.com fail > > This is not Postfix! > >> Trying 1st from @gmail.com (or any domain i've tried _other_ than 'intuit.com') > > Please see the DMARC policy of gmail.com, especially the "none" policy: > > | _dmarc.gmail.com. IN TXT "v=DMARC1; p=none; sp=quarantine; […]" This is really a security hole in gmail. Given the popularity of gmail, however, I seriously suggest somehow treating gmail as if it had p=reject, as it should. Sincerely, Demi ![]() ![]() |
> On Oct 22, 2020, at 2:11 PM, Demi M. Obenour <[hidden email]> wrote:
> > I know :( > > This is really a security hole in gmail. Given the popularity of > gmail, however, I seriously suggest somehow treating gmail as if it > had p=reject, as it should. No it should not have "p=reject" that's only for sites that only send "transactional" email. And lack of DMARC is not a "security hole". -- Viktor. |
On 10/22/20 12:25 PM, Viktor Dukhovni wrote:
>> On Oct 22, 2020, at 2:11 PM, Demi M. Obenour <[hidden email]> wrote: >> >> I know :( >> >> This is really a security hole in gmail. Given the popularity of >> gmail, however, I seriously suggest somehow treating gmail as if it >> had p=reject, as it should. > No it should not have "p=reject" that's only for sites that only send > "transactional" email. And lack of DMARC is not a "security hole". "p=quarantine" might be a better choice, but I do consider lack of DMARC to be a security hole. I certainly don't want someone to be able to forge mail that claims to be from me. There are all sorts of nasty social engineering attacks someone could do with that ability, many of which have real-world consequences. Demi ![]() ![]() |
Demi M. Obenour wrote:
> Viktor Dukhovni wrote: > >> Demi M. Obenour <[hidden email]> wrote: > >> This is really a security hole in gmail. Given the popularity of > >> gmail, however, I seriously suggest somehow treating gmail as if it > >> had p=reject, as it should. > > No it should not have "p=reject" that's only for sites that only send > > "transactional" email. And lack of DMARC is not a "security hole". > > "p=quarantine" might be a better choice, but I do consider lack of > DMARC to be a security hole. I certainly don't want someone to be > able to forge mail that claims to be from me. There are all sorts of > nasty social engineering attacks someone could do with that ability, > many of which have real-world consequences. Such as your mail from Gmail through mailing lists such as this one? DMARC breaks traditional mailing list usage because it focuses on the header address not the envelope address. Sites with a strict DMARC policy require mailing lists to either rewrite header addresses to avoid the breakage, or to drop the mail, or other worse alternatives. Strict DMARC policy is why we are often seeing "... via ..." in the From: addresses and the address rewritten now when it is coming from a site that has set a strict DMARC policy. Strict DMARC policy is suitable for banks and other direct mailing use wishing higher security but is not suitable for a user's general email where they want to send mail to mailing lists and have other interactions with the community. Bob |
On 10/22/20 3:35 PM, Bob Proulx wrote:
> Demi M. Obenour wrote: >> Viktor Dukhovni wrote: >>>> Demi M. Obenour <[hidden email]> wrote: >>>> This is really a security hole in gmail. Given the popularity of >>>> gmail, however, I seriously suggest somehow treating gmail as if it >>>> had p=reject, as it should. >>> No it should not have "p=reject" that's only for sites that only send >>> "transactional" email. And lack of DMARC is not a "security hole". >> >> "p=quarantine" might be a better choice, but I do consider lack of >> DMARC to be a security hole. I certainly don't want someone to be >> able to forge mail that claims to be from me. There are all sorts of >> nasty social engineering attacks someone could do with that ability, >> many of which have real-world consequences. > > Such as your mail from Gmail through mailing lists such as this one? > DMARC breaks traditional mailing list usage because it focuses on the > header address not the envelope address. DMARC is aimed at preventing spoofing. If someone sends a message that claims to be from me, but is not, that could damage my reputation or worse. If GMail had p=reject, such a message would be dropped as a forgery. If a relative of mine gets a message that claims to be from me, but is actually from <[hidden email]>, they at least have a chance of knowing the message is bogus. > Sites with a strict DMARC policy require mailing lists to either > rewrite header addresses to avoid the breakage, or to drop the mail, > or other worse alternatives. Strict DMARC policy is why we are often > seeing "... via ..." in the From: addresses and the address rewritten > now when it is coming from a site that has set a strict DMARC policy. To me, that is a good thing. I *want* mailing lists to either relay the message without changes, or take ownership of the message body by changing the From: header. Otherwise, they are claiming that I sent a message that I never in fact sent, which is not okay. "... via ..." is what I want to see in a mailing list message. > Strict DMARC policy is suitable for banks and other direct mailing use > wishing higher security but is not suitable for a user's general email > where they want to send mail to mailing lists and have other > interactions with the community. If a mailing list relays mail without changing it, DMARC will pass, since the digital signature will still verify correctly. Changing the message without changing the From: header is spoofing, and mailing list software that does it is broken. > Bob Sincerely, Demi ![]() ![]() |
Demi M. Obenour:
> That's because MUAs display the From: header, not the envelope address. > DMARC is aimed at preventing spoofing. If someone sends a message > that claims to be from me, but is not, that could damage my reputation > or worse. If GMail had p=reject, such a message would be dropped > as a forgery. If a relative of mine gets a message that claims to > be from me, but is actually from <[hidden email]>, > they at least have a chance of knowing the message is bogus. Enough already. Here's a From: header From: Firstname Lastname <[hidden email]> display name email address Many mail user agents, especially the GUI based ones, display the "Firstname Lastname" part, not the sender address. To see the address one has to take additional steps which many people won't take. What does this mean for ordinary users? There is a sender address that they never see, that is "secured" with DMARC and so on, but it could be total garbage because the user won't see it. What they do see is the completely unprotected "Firstname Lastname" part. Oh, and maybe an indicator that the email it is secure. Wietse |
On 22 Oct 2020, at 17:17, Wietse Venema <[hidden email]> wrote:=
> > Demi M. Obenour: >> That's because MUAs display the From: header, not the envelope address. >> DMARC is aimed at preventing spoofing. If someone sends a message >> that claims to be from me, but is not, that could damage my reputation >> or worse. If GMail had p=reject, such a message would be dropped >> as a forgery. If a relative of mine gets a message that claims to >> be from me, but is actually from <[hidden email]>, >> they at least have a chance of knowing the message is bogus. > > Enough already. Here's a From: header > > From: Firstname Lastname <[hidden email]> > display name email address > > Many mail user agents, especially the GUI based ones, display the > "Firstname Lastname" part, not the sender address. To see the address > one has to take additional steps which many people won't take. > > What does this mean for ordinary users? There is a sender address > that they never see, that is "secured" with DMARC and so on, but > it could be total garbage because the user won't see it. > > What they do see is the completely unprotected "Firstname Lastname" > part. Oh, and maybe an indicator that the email it is secure. It's worth pointing out, again, that this means that a From header in a DMARC message like: From: [hidden email] <[hidden email]> Will be shown to the user as being a secured message from "alert@#PayPal.com" -- 'You're your own worst enemy, Rincewind,' said the sword. Rincewind looked up at the grinning men. 'Bet?' --Colour of Magic |
In reply to this post by Wietse Venema
Wietse Venema wrote:
> Enough already. Here's a From: header > > From: Firstname Lastname <[hidden email]> > display name email address > > Many mail user agents, especially the GUI based ones, display the > "Firstname Lastname" part, not the sender address. To see the address > one has to take additional steps which many people won't take. You can mark me down as being really old school because I do actually see most of the interesting headers. For example in the message from you I see exactly this in the headers. (I display this set of message headers in this ordering, From: Reply-To: Resent-From: To: Cc: Mail-Followup-To: Subject: Date:) Date: Thu, 22 Oct 2020 19:17:35 -0400 (EDT) From: Wietse Venema <[hidden email]> To: Postfix users <[hidden email]> Subject: Re: sanity-check postfix XCLIENT usage ? Reply-To: Postfix users <[hidden email]> :-) > What does this mean for ordinary users? There is a sender address > that they never see, that is "secured" with DMARC and so on, but > it could be total garbage because the user won't see it. > > What they do see is the completely unprotected "Firstname Lastname" > part. Oh, and maybe an indicator that the email it is secure. I take advantage of that around April Fool's Day and send them messages from their dog and things like that. :-) > Demi M. Obenour: > > That's because MUAs display the From: header, not the envelope address. > > DMARC is aimed at preventing spoofing. If someone sends a message > > that claims to be from me, but is not, that could damage my reputation > > or worse. If GMail had p=reject, such a message would be dropped > > as a forgery. If a relative of mine gets a message that claims to > > be from me, but is actually from <[hidden email]>, > > they at least have a chance of knowing the message is bogus. The tragicomical thing is that Gmail does follow policy and when the policy of the sending site is strict DMARC and the mailing list does not rewrite then Gmail subscribers to mailing lists will get automatically unsubscribed when/if the bounce ratio exceeds the threshold! That's perhaps a surprising unintended consequence. Because the sender's message from the mailing list re-sent to the Gmail subscriber is bounced by Gmail back to the mailing list then the mailing list management software counts it as a bounce from the Gmail end against that subscriber. The sender is unmarked. So it depends upon the amount of mailing list traffic and the ratio of mail from strict DMARC sites and not strict. A large flow of non-DMARC traffic will keep the ratio low. But then if on a single day even one strict DMARC sender sends many messages causing many bounces it will trip over the threshold for the Gmail subscribers and result in all of them being unsubscribed all at once. And then a bunch of Gmail subscribers are all left wondering why they were unsubscribed. Sometimes they will even write in to the list admins asking that question. Yes these are actual events. I think the best compromise is that mailing lists must rewrite the headers when handling mail from sites with a strict DMARC policy. Although there are others that have disagreed and wished their email to be discarded rather than modified in any way. Bob |
In reply to this post by PGNet Dev
On 22/10/20 6:13 am, PGNet Dev wrote:
Before I take this up as an opendmarc question (my config &/or bug), & do more thorough digging re: intuit's published records, Two ideas:
Nick. |
In reply to this post by Bob Proulx
On 23/10/20 2:26 pm, Bob Proulx wrote:
The tragicomical thing is that Gmail does follow policy and when the policy of the sending site is strict DMARC and the mailing list does not rewrite then Gmail subscribers to mailing lists will get automatically unsubscribed when/if the bounce ratio exceeds the threshold! That's perhaps a surprising unintended consequence. Because the sender's message from the mailing list re-sent to the Gmail subscriber is bounced by Gmail back to the mailing list then the mailing list management software counts it as a bounce from the Gmail end against that subscriber. The sender is unmarked. So it depends upon the amount of mailing list traffic and the ratio of mail from strict DMARC sites and not strict. A large flow of non-DMARC traffic will keep the ratio low. But then if on a single day even one strict DMARC sender sends many messages causing many bounces it will trip over the threshold for the Gmail subscribers and result in all of them being unsubscribed all at once. And then a bunch of Gmail subscribers are all left wondering why they were unsubscribed. Sometimes they will even write in to the list admins asking that question. Yes these are actual events. I think the best compromise is that mailing lists must rewrite the headers when handling mail from sites with a strict DMARC policy. Although there are others that have disagreed and wished their email to be discarded rather than modified in any way. Provided that domains which publish strict DMARC policies sign their emails correctly with valid DKIM signatures that pass the DMARC alignment test, and the mailing list software rewrites the envelope (5321.MailFrom) sender, then the email will still pass DMARC check. The problem is sender domains that use strict DMARC policies and either don't use DKIM or fail the alignment test. FYI For a while I was emailing individuals on this mailing list when I noticed this sort of thing, and more often than not the recipient was receptive to my input, and in a position to be able to fix the issue, and was willing to do so, which is a win for everyone! Unfortunately, lately I haven't had the time... [Rhetorical question to everyone] Have you checked your
DMARC lately? ;-) Nick. |
In reply to this post by Nick Tait
On 23/10/2020 09:27, Nick Tait wrote:
+1, and when doing this you might want to redirect emails that fail DMARC into a special mailbox, something like: /etc/postfix/main.cf: /etc/postfix/milter_header_checks.pcre:
|
In reply to this post by Demi M. Obenour
On Fri, Oct 23, 2020 at 3:26 PM Demi M. Obenour <[hidden email]> wrote:
> >> "p=quarantine" might be a better choice, but I do consider lack of > >> DMARC to be a security hole. I certainly don't want someone to be > >> able to forge mail that claims to be from me. There are all sorts of > >> nasty social engineering attacks someone could do with that ability, > >> many of which have real-world consequences. Mailing lists are the most important reason why DMARC is not terribly meaningful as a trust indicator for email from gmail.com. Anyone can claim an account that looks uncannily similar to an existing account, and would fool most users unless they are particularly attentive to minute details. I've heard reports that Paypal's transactional email benefits from DMARC and that phishing is less frequent as a result. Though I'm somewhat surprised this is effective, I'm willing to believe it is actually true (the fishers must just not care enough to make a convincing pitch from a different domain). But for the consumer email providers, attestation that the email was really from Gmail is not a very strong indicator that the message is legitimate. -- Not the same Demi M. Obenour. |
Free forum by Nabble | Edit this page |