Forwarding best practices

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Forwarding best practices

John Regan
Hi,
I have a postfix-3.4.10 system being used as a relay for a subdomain where most users are forwarding their mail through it instead of sending and receiving email on it directly using a ~/.forward file and procmail.

Users can send and receive mail using their desktop email client connected through imap/submission or a webmail client.

The problem is issues like this:

825FD80D40EBC     8283 Fri Jul 31 15:00:21  [hidden email]
(host mta7.am0.yahoodns.net[98.136.96.74] said: 421 4.7.0 [TSS04] Messages from 66.104.111.99 temporarily deferred due to user complaints - 4.1
6.55.1; see https://help.yahoo.com/kb/postmaster/SLN3434.html (in reply to MAIL FROM command))

This mail server has an SPF record for itself, but no DKIM or DMARC. It also has a working reverse DNS. Mail is received by this system from two postfix relays protected with spamassassin and monitored closely.

Yahoo recommends messages are DKIM signed, but we were concerned about the effect mailing lists and other emails would have being forwarded through the server.

We're also using two transport filters to deliver mail:
polite unix - - n - - smtp
        -o syslog_name=postfix-polite
turtle unix - - n - - smtp
        -o syslog_name=postfix-turtle

We're specifying domains like yahoo to go through the turtle transport:
/yahoo\.com$/ turtle:
/yahoo(\.[a-z]{2,3}){1,2}$/ turtle:

Perhaps that practice has changed and DKIM and DMARC should be implemented on relays now?

Can someone recommend a set of best practices for using postfix to relay mail to yahoo/gmail in this way?



Reply | Threaded
Open this post in threaded view
|

Re: Forwarding best practices

@lbutlr
On 31 Jul 2020, at 14:18, John Regan <[hidden email]> wrote:
> This mail server has an SPF record for itself, but no DKIM or DMARC. It also has a working reverse DNS. Mail is received by this system from two postfix relays protected with spamassassin and monitored closely.

Yahoo doesn’t care, and IME will reject mail regardless of settings on your end based on any single moron files a message as spam, unless you are a large enough mailer that they actually care and have whitelisted you. That's basically only google AFAICT.

> Can someone recommend a set of best practices for using postfix to relay mail to yahoo/gmail in this way?

You can't fix Yahoo.



--
Hey, how come Andrew gets to get up? If he gets up, we'll all get up!
        It'll be anarchy!

Reply | Threaded
Open this post in threaded view
|

Re: Forwarding best practices

Bob Proulx
In reply to this post by John Regan
John Regan wrote:
> Subject: Forwarding best practices
...
> Can someone recommend a set of best practices for using postfix to relay
> mail to yahoo/gmail in this way?

The Best Practice for forwarding today is not to do it.  It has long
been a friendly allowed practice on the net.  But as Yahoo, Google,
Microsoft, others, become the 800lb gorillas driving things then
nothing is traditional or standard anymore.  Now it is the rule of
might makes right.  And they are the ones with the might.

No matter what you do on your end there is no way to guarentee that
the large mailbox providers will accept the forwarded messages.
Because at any point in time any of those users might click "Spam" on
the message.  And there is no way you can prevent this.  It's a human
problem of human training.  You can't stop them from doing this bad
thing on messages that were forwarded.  It's a bad thing because they
are receiving the message via a forward and clicking on Spam punishes
the forwarder.  Punishes *your* forwarding site.

It might be an actual spam message that has slipped through your
upstream anti-spam filtering.  A message that your site should have
rejected at SMTP time but did not.  It slips through.  It was a false
positive.  No anti-spam system is 100% accurate and correct.  Then
your site SMTP transfers it to the large mailbox provider.  It's spam.
The user who set up the forwarding clicks Spam on it.  BAM!  You are
now listed as a spammer!  (Or at least given one demerit for this
particular email.)  Because your site was the last one with the
message.  Clearly your site sent that spam.  It's done.  Black eye for
you.  Hopefully it will heal up before you get another one.

My experience is that if it is Microsoft that they will allow one free
black eye for you.  I contact them and say, hey, what's the data for
this so I can improve things?  What message do you have in the corpus
of evidence for me?  I want to figure out what is happening and stop
it.  They write me back and say, "Don't talk to me you spammer.  We
will show you nothing.  But we will delist you this one time."
Obviously this is a paraphrase from memory.

Now delisted things work okay again for a while.  Then for reasons I
have no idea there suddenly Microsoft rejects all mail again.  I try
the official contact channels.  Now they go, "Don't talk to me you
spammer, you are a repeat offender, we are not going to show you
anything, and we are not talking to you again."  Paraphrasing again.

I have yet to see any evidence from Microsoft as to what messages they
think are worthy of being IP blocked regardless of my attempts to
communicate with them.  Therefore I have no way to improve processes
on my system.  I am thinking one of my users is forwarding and then
reporting messages as spam.  But without data I can't be sure.
Without data I have nothing to grip upon.  I don't know anything firm
about who or what.  Eventually I am forced to route Microsoft
destinations through a different IP address to avoid the IP block.
They have the might and I do not.

> 825FD80D40EBC     8283 Fri Jul 31 15:00:21  [hidden email]
> (host mta7.am0.yahoodns.net[98.136.96.74] said: 421 4.7.0 [TSS04] Messages
> from 66.104.111.99 temporarily deferred due to user complaints - 4.1
> 6.55.1; see https://help.yahoo.com/kb/postmaster/SLN3434.html (in reply to
> MAIL FROM command))

Fortunately for the above it is only a 4xx temporary code.  They are
only rate limiting you.  It's not a hard block.  So examine what
messages are in the queue and determine if any of them are spam.
Clean the queue as appropriate.  Then wait and hope that the rate
limit is all that happens.

> This mail server has an SPF record for itself, but no DKIM or DMARC. It
> also has a working reverse DNS. Mail is received by this system from two
> postfix relays protected with spamassassin and monitored closely.

I recommend setting up SPF and DKIM.  Fortunately those are fairly
straight forward.  I recommend *not* setting up DMARC unless you are a
bank or other financial institution or other higher security sender.
Strict DMARC outright prevents forwarding in all forms.  And other
things.  It's not appropriate for a general use email site.  It's
appropriate for higher security outgoing email sites such as financial
institution that want to prevent forgeries and would not normally be
using mailing lists or other general use email.

> Yahoo recommends messages are DKIM signed, but we were concerned about the
> effect mailing lists and other emails would have being forwarded through
> the server.

AFAIK use of DKIM and mailing lists is not problematic.  However
strict DMARC completely breaks mailing lists.  You have probably been
seeing the from headers changing for people sending to mailing lists
from strict DMARC sites now.  Whereas before it would list their
sending address now that is rejected at receiving sites forcing it to
be re-sent and now the header typically says the name plus "via the
mailing list" with the mailing list as the from address.  That's a
workaround for the sender mailing from a strict DMARC site.  Because
if the workaround is not implemented then a list of bad things happen
that seems at first glance to be unrelated.

The typical symptom of not mangling from addresses for mailing lists
is that entire sets of subscribers at, say, Gmail are automatically
unsubscribed due to too many bounces from Gmail.  The reason?
Because, say, zoho.eu has set a strict DMARC policy it means that
Google will reject messages that violate the sender's policy.  Which
is as it should be.  But now anyone sending from zoho.eu cannot mail
through a mailing list.  Not without mangling.  Any mailing list that
does not mangle will find Google rejecting those messages.  Which
arrive as a bounce rejection to the mailing list management software.
Which will think the recipient at Gmail is simple failing and will
eventually automatically unsubscribe the failing address.  Which at
first glance seems unrelated to the root cause since the people being
unsubscribed are @gmail.com addresses but the problem is @zoho.eu
addresses being rejected.  Therefore mailing lists must now avoid the
problem.  There are several different strategies being used.  Mangling
the from address to be via the mailing list seems the Best Practice
now.  And note this DMARC issue is completely separate from the IP
reputation due to spam forwarding issue.

But of course I assume that your site has been allowing forwarding for
a long time.  It's now part of the status quo there.  Changing that
would be very disruptive.  I understand that completely.  It's a
problem.  A problem without any good solutions.  I can only suggest
that you be aware that it is "thin ice" and keep looking for a
solution.  Along with the rest of us.

Bob
Reply | Threaded
Open this post in threaded view
|

Re: Forwarding best practices

Jaroslaw Rafa
Dnia  5.08.2020 o godz. 14:23:00 Bob Proulx pisze:
>
> The Best Practice for forwarding today is not to do it.  It has long
> been a friendly allowed practice on the net.  But as Yahoo, Google,
> Microsoft, others, become the 800lb gorillas driving things then
> nothing is traditional or standard anymore.  Now it is the rule of
> might makes right.  And they are the ones with the might.
[...]
> But of course I assume that your site has been allowing forwarding for
> a long time.  It's now part of the status quo there.  Changing that
> would be very disruptive.  I understand that completely.  It's a
> problem.  A problem without any good solutions.  I can only suggest
> that you be aware that it is "thin ice" and keep looking for a
> solution.  Along with the rest of us.

What's ironic in all this mess is that Gmail itself gives the users option
to forward mail to a different address...
--
Regards,
   Jaroslaw Rafa
   [hidden email]
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."
Reply | Threaded
Open this post in threaded view
|

Re: Forwarding best practices

@lbutlr
On 06 Aug 2020, at 04:32, Jaroslaw Rafa <[hidden email]> wrote:

> Dnia  5.08.2020 o godz. 14:23:00 Bob Proulx pisze:
>>
>> The Best Practice for forwarding today is not to do it.  It has long
>> been a friendly allowed practice on the net.  But as Yahoo, Google,
>> Microsoft, others, become the 800lb gorillas driving things then
>> nothing is traditional or standard anymore.  Now it is the rule of
>> might makes right.  And they are the ones with the might.
> [...]
>> But of course I assume that your site has been allowing forwarding for
>> a long time.  It's now part of the status quo there.  Changing that
>> would be very disruptive.  I understand that completely.  It's a
>> problem.  A problem without any good solutions.  I can only suggest
>> that you be aware that it is "thin ice" and keep looking for a
>> solution.  Along with the rest of us.
>
> What's ironic in all this mess is that Gmail itself gives the users option
> to forward mail to a different address...

Doubly ironic since gmail is one of the worst services for ACCEPTING forwarded mail.



--
"Are you pondering what I'm pondering?"
"I think so, Brain. But would the villains really have gotten away
        with it, if it weren't for those pesky kids and their dog?"

Reply | Threaded
Open this post in threaded view
|

Re: Forwarding best practices

Kris Deugau
In reply to this post by Bob Proulx
Bob Proulx wrote:

> No matter what you do on your end there is no way to guarentee that
> the large mailbox providers will accept the <del>forwarded</del> messages.

FTFY. :(

> Because at any point in time any of those users might click "Spam" on
> the message.  And there is no way you can prevent this.  It's a human
> problem of human training.

It's not even about forwarded mail, or spam.

I'm plugged in to Microsoft's "Junk Mail Reporting Program" for mail
coming from both our outbound cluster and forwarded from our MX cluster.

End users on Hotmail/Outlook.com routinely mark *legitimate direct
messages in ongoing conversations* as spam.  They report *legitimate
government communications about something they've asked a government
office about*.  They flag *mail from their lawyer*.

I can only guess that the "Mark as spam" button looks a lot like a
"Delete" symbol of some kind, or I have to give up all hope of
intelligence on the part of end users.

> My experience is that if it is Microsoft that they will allow one free
> black eye for you.  I contact them and say, hey, what's the data for
> this so I can improve things?  What message do you have in the corpus
> of evidence for me?  I want to figure out what is happening and stop
> it.  They write me back and say, "Don't talk to me you spammer.  We
> will show you nothing.  But we will delist you this one time."
> Obviously this is a paraphrase from memory.
>
> Now delisted things work okay again for a while.  Then for reasons I
> have no idea there suddenly Microsoft rejects all mail again.  I try
> the official contact channels.  Now they go, "Don't talk to me you
> spammer, you are a repeat offender, we are not going to show you
> anything, and we are not talking to you again."  Paraphrasing again.

Or they refer you to their rules for "bulk senders", which are variously
inapplicable or utterly irrelevant for both forwarded mail and ISP-type
outbound relay mail flow.

> I have yet to see any evidence from Microsoft as to what messages they
> think are worthy of being IP blocked regardless of my attempts to
> communicate with them.

If you can get recognized as a suitable contact for your outbound IPs
with Microsoft's SNDS and JMRP you can start seeing (some of?) the mail
marked as spam.  It's helped us locate low-volume compromised accounts
now and then.

If you get IP-blocked, you're sending/forwarding "too much spam"
(whatever that means), and in the end it may just come down to telling
your users they can't forward mail to Hotmail accounts any more as a
matter of policy.

   Therefore I have no way to improve processes
> on my system.  I am thinking one of my users is forwarding and then
> reporting messages as spam.  But without data I can't be sure.
> Without data I have nothing to grip upon.  I don't know anything firm
> about who or what.  Eventually I am forced to route Microsoft
> destinations through a different IP address to avoid the IP block.
> They have the might and I do not.

Sometimes it seems like they're blocking because it's a day ending in y.

-kgd
Reply | Threaded
Open this post in threaded view
|

Re: Forwarding best practices

Harald Koch-2
On Thu, Aug 6, 2020, at 09:53, Kris Deugau wrote:
>
> I can only guess that the "Mark as spam" button looks a lot like a
> "Delete" symbol of some kind, or I have to give up all hope of
> intelligence on the part of end users.

In many of the email programs I use, J/K are the "next message/previous message" keyboard shortcuts.

In Outlook, J is the "Mark as Junk" shortcut.

I swear I hit it about once a day as I'm switching email clients ...

--
Harald Koch
[hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Forwarding best practices

Miles Fidelman
In reply to this post by Jaroslaw Rafa

>> The Best Practice for forwarding today is not to do it.  It has long
>> been a friendly allowed practice on the net.  But as Yahoo, Google,
>> Microsoft, others, become the 800lb gorillas driving things then
>> nothing is traditional or standard anymore.  Now it is the rule of
>> might makes right.  And they are the ones with the might.

Ummm.... NOT forwarding translates to vendor lock-in.

IMO, the best practice for forwarding, today, is to offer an
email-address-for-life, the way many alumni & professional associations
do.  Then again, I'm a firm believer in managing my own domains, and not
putting myself in the position where a vendor can lock me in.

Miles Fidelman

--
In theory, there is no difference between theory and practice.
In practice, there is.  .... Yogi Berra

Theory is when you know everything but nothing works.
Practice is when everything works but no one knows why.
In our lab, theory and practice are combined:
nothing works and no one knows why.  ... unknown