How to avoid being classified as spam by Google?

classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|

How to avoid being classified as spam by Google?

martin f krafft-2
Folks,

I hope this is not too off-topic, but I figure this is the best
mailing list because we're probably not in this boat alone, wherein
we're annoyed (very) and a bit helpless about Google. I have to ask
here, because Google of course doesn't care about us.

We operate several postfix mailservers with hundreds of users, and
the rate at which these users find their emails spamfiled on the
side of Gmail recipients is increasing.

And yet, we're doing everything we can! SPF is configured, DMARC is
configured, DKIM works. And we've even registered the domains and
MXs with Google's Postmaster services,
https://postmaster.google.com/.

And yet, most mails are filed as spam on their side. And we just
cannot figure out why. I promise we're not writing about
enlargements or crypto scams. We use proper English (or German), and
we know how to capitalise words.

Looking at messages being classified, there is nothing in the
headers that gives any clues. In fact, the headers look just fine:

  Authentication-Results: mx.google.com; dkim=pass (test mode)
    header.i=@example.org header.s=2015-11-14 header.b=T7jbyqDv;
    spf=pass (google.com: best guess record for domain of
    [hidden email] designates 2001:db8:bad::cafe:: as
    permitted sender) smtp.mailfrom=[hidden email];
    dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=example.org

What else can we do? How do you deal with this problem?

--
@martinkrafft | <a href="https://riot.im/app/#/room/#madduck:madduck.net">https://riot.im/app/#/room/#madduck:madduck.net
 
"for art to exist, for any sort of aesthetic activity or perception to
 exist, a certain physiological precondition is indispensable:
 intoxication."
                                              -- friedrich nietzsche
 
spamtraps: [hidden email]

digital_signature_gpg.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: How to avoid being classified as spam by Google?

Benny Pedersen-2
martin f krafft skrev den 2019-10-06 18:36:

>  Authentication-Results: mx.google.com; dkim=pass (test mode)
>    header.i=@example.org header.s=2015-11-14 header.b=T7jbyqDv;
>    spf=pass (google.com: best guess record for domain of
>    [hidden email] designates 2001:db8:bad::cafe:: as
>    permitted sender) smtp.mailfrom=[hidden email];
>    dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=example.org
>
> What else can we do? How do you deal with this problem?

dkkim running in test mode ?, see if domain have t= in dns

just guesssing
Reply | Threaded
Open this post in threaded view
|

Re: How to avoid being classified as spam by Google?

Bill Cole-3
In reply to this post by martin f krafft-2
On 6 Oct 2019, at 12:36, martin f krafft wrote:

> Folks,
>
> I hope this is not too off-topic, but I figure this is the best
> mailing list because we're probably not in this boat alone, wherein
> we're annoyed (very) and a bit helpless about Google. I have to ask
> here, because Google of course doesn't care about us.

The MailOp list is probably a better choice:
https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Actual Google mail admins respond to such queries there. Really.

--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Reply | Threaded
Open this post in threaded view
|

Re: How to avoid being classified as spam by Google?

Benny Pedersen-2
Bill Cole skrev den 2019-10-06 18:50:

> The MailOp list is probably a better choice:
> https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

NET::ERR_CERT_SYMANTEC_LEGACY

> Actual Google mail admins respond to such queries there. Really.

if recipient keeps mails in spam folder how could google then classifiy
it better ?
Reply | Threaded
Open this post in threaded view
|

Re: How to avoid being classified as spam by Google?

martin f krafft-2
In reply to this post by Benny Pedersen-2
Quoting "Benny Pedersen", who wrote on 2019-10-06 at 18:44 Uhr +0200:
>dkkim running in test mode ?, see if domain have t= in dns

Yes, on some domains it's still running in test mode. Is that enough
reason for Google admins to flick us the finger?

--
@martinkrafft | <a href="https://riot.im/app/#/room/#madduck:madduck.net">https://riot.im/app/#/room/#madduck:madduck.net
 
"god is a comedian playing to an audience too afraid to laugh."
                                                         -- voltaire
 
spamtraps: [hidden email]

digital_signature_gpg.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: How to avoid being classified as spam by Google?

martin f krafft-2
In reply to this post by Bill Cole-3
Quoting "Bill Cole", who wrote on 2019-10-06 at 12:50 Uhr -0400:
>The MailOp list is probably a better choice:
>https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

Thanks! I didn't know about that. Will re-ask there. Sorry for the
noise.

--
@martinkrafft | <a href="https://riot.im/app/#/room/#madduck:madduck.net">https://riot.im/app/#/room/#madduck:madduck.net
 
"anyone who is capable of getting themselves made president
 should on no account be allowed to do the job"
                                                    -- douglas adams
 
spamtraps: [hidden email]

digital_signature_gpg.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: How to avoid being classified as spam by Google?

Benny Pedersen-2
martin f krafft skrev den 2019-10-06 19:03:
> Quoting "Bill Cole", who wrote on 2019-10-06 at 12:50 Uhr -0400:
>> The MailOp list is probably a better choice:
>> https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop
>
> Thanks! I didn't know about that. Will re-ask there. Sorry for the
> noise.

note NET::ERR_CERT_SYMANTEC_LEGACY
Reply | Threaded
Open this post in threaded view
|

Re: How to avoid being classified as spam by Google?

Jaroslaw Rafa
In reply to this post by Bill Cole-3
Dnia  6.10.2019 o godz. 12:50:27 Bill Cole pisze:
>
> The MailOp list is probably a better choice:
> https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop

I have the very same issue as the OP, thanks for pointing to that list!
--
Regards,
   Jaroslaw Rafa
   [hidden email]
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."
Reply | Threaded
Open this post in threaded view
|

Re: How to avoid being classified as spam by Google?

Wietse Venema
In reply to this post by martin f krafft-2
martin f krafft:
>   Authentication-Results: mx.google.com; dkim=pass (test mode)
>     header.i=@example.org header.s=2015-11-14 header.b=T7jbyqDv;
>     spf=pass (google.com: best guess record for domain of
>     [hidden email] designates 2001:db8:bad::cafe:: as
>     permitted sender) smtp.mailfrom=[hidden email];
>     dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=example.org
>
> What else can we do? How do you deal with this problem?

Perhaps the SMTP client IP address 2001:db8:bad::cafe:: has no PTR
record (or the name does not resolve to 2001:db8:bad::cafe::).

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: How to avoid being classified as spam by Google?

martin f krafft-2
Quoting "Wietse Venema", who wrote on 2019-10-06 at 19:13 Uhr -0400:
>Perhaps the SMTP client IP address 2001:db8:bad::cafe:: has no PTR
>record (or the name does not resolve to 2001:db8:bad::cafe::).

Good point, but the address has a PTR record to a name with an AAAA
record pointing to the address.

--
@martinkrafft | <a href="https://riot.im/app/#/room/#madduck:madduck.net">https://riot.im/app/#/room/#madduck:madduck.net
 
kermit: why are there so many songs about rainbows?
fozzy: that's part of what rainbows do.
 
spamtraps: [hidden email]

digital_signature_gpg.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: How to avoid being classified as spam by Google?

allenc


On 07/10/2019 06:11, martin f krafft wrote:
> Quoting "Wietse Venema", who wrote on 2019-10-06 at 19:13 Uhr -0400:
>> Perhaps the SMTP client IP address 2001:db8:bad::cafe:: has no PTR record (or
>> the name does not resolve to 2001:db8:bad::cafe::).
>
> Good point, but the address has a PTR record to a name with an AAAA record
> pointing to the address.
>
Only one set of double-colons is allowed in an IPv6 address.   It expands to an
unspecified number of zeros; doing it twice results in ambiguity.

Allen C
Reply | Threaded
Open this post in threaded view
|

Re: How to avoid being classified as spam by Google?

Peter Ajamian
In reply to this post by martin f krafft-2
On 7/10/19 5:36 AM, martin f krafft wrote:

> Folks,
>
> I hope this is not too off-topic, but I figure this is the best mailing
> list because we're probably not in this boat alone, wherein we're
> annoyed (very) and a bit helpless about Google. I have to ask here,
> because Google of course doesn't care about us.
>
> We operate several postfix mailservers with hundreds of users, and the
> rate at which these users find their emails spamfiled on the side of
> Gmail recipients is increasing.
>
> And yet, we're doing everything we can! SPF is configured, DMARC is
> configured, DKIM works. And we've even registered the domains and MXs
> with Google's Postmaster services, https://postmaster.google.com/.

We get this question on IRC a lot as well, it's a common problem.  The
generic answer I always give is this:

If you're having problems getting your mail received by major ESPs you
should first check your fcrdns*, then make sure you have SPF, DKIM and
DMARC configured.** Sign up for the dnswl at http://www.dnswl.org. Also
sign up for ESP-specific programs such as feedback loops, Google
postmaster tools and Microsoft's SNDS.  Check the individual postmaster
pages for each ESP that you're having problems with to make sure that
you're in compliance with all of their policies and if you still have
problems contact the support for the ESP.  Also check to make sure
you're not on any DNSRBLs and submit removal requests from any you
appear on.***

* http://en.wikipedia.org/wiki/Forward_Confirmed_reverse_DNS: your IP
address should resolve to $myhostname, which in turn should resolve back
to your IP.

** If you just need a DMARC policy to help satisfy ESP recommendations
you can use this to basically tell servers not to enforce DMARC on your
mail: _dmarc.example.com. TXT "v=DMARC1;p=none;adkim=r;aspf=r;pct=0"

*** http://multirbl.valli.org/

... I know that you're at least doing some of these things from your
message, but I would recommend that you go through the above carefully
and do everything listed before you attempt to contact the ESP services
for help.


Peter
Reply | Threaded
Open this post in threaded view
|

Re: How to avoid being classified as spam by Google?

Wietse Venema
In reply to this post by martin f krafft-2
martin f krafft:
> Quoting "Wietse Venema", who wrote on 2019-10-06 at 19:13 Uhr -0400:
> >Perhaps the SMTP client IP address 2001:db8:bad::cafe:: has no PTR
> >record (or the name does not resolve to 2001:db8:bad::cafe::).
>
> Good point, but the address has a PTR record to a name with an AAAA
> record pointing to the address.

The form "2001:db8:bad::cafe::" does not look kosher to me.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: How to avoid being classified as spam by Google?

Jaroslaw Rafa
In reply to this post by Peter Ajamian
Dnia  7.10.2019 o godz. 23:54:41 Peter pisze:
> Also sign up for ESP-specific programs such as
> feedback loops, Google postmaster tools and Microsoft's SNDS. Check
> the individual postmaster pages for each ESP that you're having
> problems with to make sure that you're in compliance with all of
> their policies and if you still have problems contact the support
> for the ESP.

Google Postmaster Tools won't give you any information unless you send at
least a few hundred messages a day to Gmail users. So it's of no use for
small servers like mine that serve only a few accounts and send only a
handful of messages a day (and there are days when no message is sent).

> ** If you just need a DMARC policy to help satisfy ESP
> recommendations you can use this to basically tell servers not to
> enforce DMARC on your mail: _dmarc.example.com. TXT
> "v=DMARC1;p=none;adkim=r;aspf=r;pct=0"

According to specs, you can't set pct to 0, it must be at least 1.
--
Regards,
   Jaroslaw Rafa
   [hidden email]
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."
Reply | Threaded
Open this post in threaded view
|

Re: How to avoid being classified as spam by Google?

Peter Ajamian
In reply to this post by Peter Ajamian
On 8/10/19 12:04 AM, Jaroslaw Rafa wrote:

> Dnia  7.10.2019 o godz. 23:54:41 Peter pisze:
>> Also sign up for ESP-specific programs such as
>> feedback loops, Google postmaster tools and Microsoft's SNDS. Check
>> the individual postmaster pages for each ESP that you're having
>> problems with to make sure that you're in compliance with all of
>> their policies and if you still have problems contact the support
>> for the ESP.
>
> Google Postmaster Tools won't give you any information unless you send at
> least a few hundred messages a day to Gmail users. So it's of no use for
> small servers like mine that serve only a few accounts and send only a
> handful of messages a day (and there are days when no message is sent).

The OP said, "We operate several postfix mailservers with hundreds of
users", so they would qualify for this.

>> ** If you just need a DMARC policy to help satisfy ESP
>> recommendations you can use this to basically tell servers not to
>> enforce DMARC on your mail: _dmarc.example.com. TXT
>> "v=DMARC1;p=none;adkim=r;aspf=r;pct=0"
>
> According to specs, you can't set pct to 0, it must be at least 1.

RFC 7489 section 6.3: pct:  (plain-text integer between 0 and 100...


Peter
Reply | Threaded
Open this post in threaded view
|

Re: How to avoid being classified as spam by Google?

Jaroslaw Rafa
In reply to this post by Peter Ajamian
Dnia  7.10.2019 o godz. 23:54:41 Peter pisze:

>
> We get this question on IRC a lot as well, it's a common problem.
> The generic answer I always give is this:
>
> If you're having problems getting your mail received by major ESPs
> you should first check your fcrdns*, then make sure you have SPF,
> DKIM and DMARC configured.** Sign up for the dnswl at
> http://www.dnswl.org. Also sign up for ESP-specific programs such as
> feedback loops, Google postmaster tools and Microsoft's SNDS.  Check
> the individual postmaster pages for each ESP that you're having
> problems with to make sure that you're in compliance with all of
> their policies and if you still have problems contact the support
> for the ESP.  Also check to make sure you're not on any DNSRBLs and
> submit removal requests from any you appear on.***

Well, but let's think about it for a while.

The very meaning of the word "spam" refers to *content* of the message, not
to the way the sending mailserver is configured. It doesn't matter where the
message is coming from and how it was sent, the content - and content only -
decides if it is spam or not spam.
So, if an email that has perfectly legitimate content and not any signs of
spam, ends up in spam folder because of the way the sending mailserver is
configured, it means that the configuration of the spam filter on the
receiving end is obviously *wrong*. Of course, the spam filters can - and do
- use SPF, DNS forward-reverse match, blacklists etc. as *hints* if the
message can potentially be spam, but in the end the content analysis should
have the highest weight in spam-score for the message. If the spam filters
pay more attention to things like SPF, DKIM etc. than to the content, then
something's certainly wrong with those spam filters, and not with the
sender.

And we should say it loud and clear, say it publicly, and repeat it as often
as we can and everywhere we can. This is the only way to make those big
companies see users' dissatisfaction about their policies (not *their*
users, but Internet users in general) and possibly force them to improve
their spam filters.

I was always taught that the main goal in spam filtering is of course filter
as much spam as possible, *but* at the same time keeping the number of the
errors of second kind (ie. non-spam message being mis-classified as spam) at
"virtual zero" level. Errors of second kind are much bigger problem than
errors of first kind (ie. not catching an actual spam), because if user
accidentally gets one or two spam messages in his/her inbox, he/she can
easily delete them; but if a legitimate message is accidentally put into
spam folder, the user will probably never see it, as most users don't look
into their spam folders at all. And if the user doesn't look there and
doesn't correct the mis-classification, the algorithm will still "think" the
message is really a spam. Thus, next messages from the same sender have
higher chance of being wrongly marked as spam as well.

Looks like those big e-mail companies don't care at all about minimizing the
number of errors of second kind. This makes e-mail unreliable as a means of
communications. You could never know if the message you sent reaches the
recipient or not. And when you learn that it hasn't it may be too late,
because eg. the event you were asking about has already happened.
--
Regards,
   Jaroslaw Rafa
   [hidden email]
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."
Reply | Threaded
Open this post in threaded view
|

Re: How to avoid being classified as spam by Google?

Wietse Venema
Jaroslaw Rafa:

> Dnia  7.10.2019 o godz. 23:54:41 Peter pisze:
> >
> > We get this question on IRC a lot as well, it's a common problem.
> > The generic answer I always give is this:
> >
> > If you're having problems getting your mail received by major ESPs
> > you should first check your fcrdns*, then make sure you have SPF,
> > DKIM and DMARC configured.** Sign up for the dnswl at
> > http://www.dnswl.org. Also sign up for ESP-specific programs such as
> > feedback loops, Google postmaster tools and Microsoft's SNDS.  Check
> > the individual postmaster pages for each ESP that you're having
> > problems with to make sure that you're in compliance with all of
> > their policies and if you still have problems contact the support
> > for the ESP.  Also check to make sure you're not on any DNSRBLs and
> > submit removal requests from any you appear on.***
>
> Well, but let's think about it for a while.
>
> The very meaning of the word "spam" refers to *content* of the message, not
> to the way the sending mailserver is configured. It doesn't matter where the
> message is coming from and how it was sent, the content - and content only -
> decides if it is spam or not spam.

Can we keep this about POSTFIX?

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: How to avoid being classified as spam by Google?

Robert Schetterer-2
In reply to this post by martin f krafft-2
Am 07.10.19 um 07:11 schrieb martin f krafft:
> Quoting "Wietse Venema", who wrote on 2019-10-06 at 19:13 Uhr -0400:
>> Perhaps the SMTP client IP address 2001:db8:bad::cafe:: has no PTR
>> record (or the name does not resolve to 2001:db8:bad::cafe::).
>
> Good point, but the address has a PTR record to a name with an AAAA
> record pointing to the address.
>

Also a wide bug is not to include the ipv6 stuff in SPF, did you checked
this, in the past creating a extra transport for google only via ipv4
was helpfull too

--
[*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64
Schleißheimer Straße 26/MG, 80333 München

Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
Reply | Threaded
Open this post in threaded view
|

Re: How to avoid being classified as spam by Google?

martin f krafft-2
In reply to this post by Wietse Venema
Quoting "Allen Coates", who wrote on 2019-10-07 at 10:15 Uhr +0100:
>Only one set of double-colons is allowed in an IPv6 address.   It expands to an
>unspecified number of zeros; doing it twice results in ambiguity.

Quoting "Wietse Venema", who wrote on 2019-10-07 at 07:00 Uhr -0400:
>The form "2001:db8:bad::cafe::" does not look kosher to me.

You guys are good!

2001:db8::/32 is the "example.org" of IPv6 addresses, and the
address I posted is made up.

One of the actual addresses in question is 2001:a60:902f::bcae:fda6
which resolves to ambassador.madduck.net., which resolves to
2001:a60:902f::bcae:fda6. So I think I can tick off the "fcrdns"
requirement.

I really appreciate all your eyeballs. I really do!

--
@martinkrafft | <a href="https://riot.im/app/#/room/#madduck:madduck.net">https://riot.im/app/#/room/#madduck:madduck.net
 
"fools ignore complexity. pragmatics suffer it.
 some can avoid it. geniuses remove it."
                                                      -- alan perlis
 
spamtraps: [hidden email]

digital_signature_gpg.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: How to avoid being classified as spam by Google?

Benny Pedersen-2
In reply to this post by Robert Schetterer-2
Robert Schetterer skrev den 2019-10-07 18:21:

> Also a wide bug is not to include the ipv6 stuff in SPF, did you
> checked this, in the past creating a extra transport for google only
> via ipv4 was helpfull too

amazon have around 800000 ipv4 adresses in there spf, is this nearly as
good as +all :(

why spf is not have a max ip in there rfc :(

gmail is 2nd place there

https://dmarcian.com/spf-survey/?domain=gmail.com

its more important that spf is maked right then fight ipv6 ipv4 problems
12