Microsoft silently discarding emails after recepit

classic Classic list List threaded Threaded
35 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Microsoft silently discarding emails after recepit

Yuval Levy
Hello list,

I am relatively confident that the topic below has already been
discussed somewhere, but I was not able to find anything in this list's
archives nor on the internet at large and I have not been able to solve
the issue:  Microsoft's outlook.com seems to silently discard emails
that I send from two different SMTP servers:

(A) self-operated Postfix on a Digital Ocean droplet, has been in
operation for a few years, no complaints or problems, low traffic
(average less than 500 emails/day).  SPF/DKIM/DMARC/PTR anything that I
can think of is properly set up and scores 10/10 on
https://www.mail-tester.com/

Logs:

Jan  4 23:43:13 mx postfix/smtp[29713]: Host offered STARTTLS:
[hotmail-ca.olc.protection.outlook.com]
Jan  4 23:43:13 mx postfix/smtp[29713]: ABD28C03F9:
to=<[hidden email]>,
relay=hotmail-ca.olc.protection.outlook.com[104.47.54.33]:25, delay=1.2,
delays=0.34/0.01/0.24/0.6, dsn=2.6.0, status=sent (250 2.6.0
<[hidden email]>
[InternalId=3461743652204,
Hostname=DM3NAM06HT901.Eop-nam06.prod.protection.outlook.com] 138931
bytes in 0.359, 377.884 KB/sec Queued mail for delivery)

(B) my university's alumni email, operated by Google.  I have no logs
for it, but I am confident Google's servers are OK.

In both cases I am given the impression that my email has been "queued
for delivery" or otherwise accepted, BUT...

(1) Recipient: a person that I never corresponded with before: does not
receive anything, neither from (A) nor from (B)

(2) Recipient: an outlook.com mailbox that I keep for test purposes:
does not receive from (A) nor from (B), but does receive replies from
(A) in response to an email that was first sent from outlook.com to (A)

(3) Recipient: a person in a large organization that has switched about
a year ago to office365 and with whom I have weekly email exchanges:
receives from (A) and (B), no problem.

If there was at least an indication of rejection, I could try to contact
my correspondents otherwise.  Instead, Microsoft tells my server that
everything is OK and then drops the ball, leaving sender and recipient
in the dark.  In my view, Microsoft's conduct is unacceptable.  If it
does not deliver an email, it should at least notify the sender SMTP?

Have other SMTP operators been confronted with this?  And what have you
done about it, if anything?  Is there anything I can do to improve my
self-operated SMTP server to increase likelihood of delivery through
Microsoft?  Has anybody been able to engage with Microsoft on the
subject?  How, through what point of contact, and to what result?

Thanks,

Yuval Levy, JD, MBA, CFA
Ontario-licensed lawyer
Reply | Threaded
Open this post in threaded view
|

Re: Microsoft silently discarding emails after recepit

soumitri@iitk.ac.in
I am facing the same issue with hotmail, where Microsoft possibly
discards all mails with out showing ant error or reason for one of my
outgoing relay.

hotmail users are not receiving the mail. May be microsoft has a very
tight TLS security check (which work with all other domains).

------------------

relay=hotmail-com.olc.protection.outlook.com[104.47.2.33]:25, delay=6.6,
delays=0.09/0.05/3.8/2.6, dsn=2.6.0, status=sent (250 2.6.0
<[hidden email]>
[InternalId=74934294472548,
Hostname=DB5EUR01HT155.eop-EUR01.prod.protection.outlook.com] 11943
bytes in 0.271, 42.991 KB/sec Queued mail for delivery)

-----------------------------------------

Sincerely,
Soumitri Mishra, IITK

On 06/01/18 3:24 AM, Yuval Levy wrote:

> Hello list,
>
> I am relatively confident that the topic below has already been
> discussed somewhere, but I was not able to find anything in this list's
> archives nor on the internet at large and I have not been able to solve
> the issue:  Microsoft's outlook.com seems to silently discard emails
> that I send from two different SMTP servers:
>
> (A) self-operated Postfix on a Digital Ocean droplet, has been in
> operation for a few years, no complaints or problems, low traffic
> (average less than 500 emails/day).  SPF/DKIM/DMARC/PTR anything that I
> can think of is properly set up and scores 10/10 on
> https://www.mail-tester.com/
>
> Logs:
>
> Jan  4 23:43:13 mx postfix/smtp[29713]: Host offered STARTTLS:
> [hotmail-ca.olc.protection.outlook.com]
> Jan  4 23:43:13 mx postfix/smtp[29713]: ABD28C03F9:
> to=<[hidden email]>,
> relay=hotmail-ca.olc.protection.outlook.com[104.47.54.33]:25, delay=1.2,
> delays=0.34/0.01/0.24/0.6, dsn=2.6.0, status=sent (250 2.6.0
> <[hidden email]>
> [InternalId=3461743652204,
> Hostname=DM3NAM06HT901.Eop-nam06.prod.protection.outlook.com] 138931
> bytes in 0.359, 377.884 KB/sec Queued mail for delivery)
>
> (B) my university's alumni email, operated by Google.  I have no logs
> for it, but I am confident Google's servers are OK.
>
> In both cases I am given the impression that my email has been "queued
> for delivery" or otherwise accepted, BUT...
>
> (1) Recipient: a person that I never corresponded with before: does not
> receive anything, neither from (A) nor from (B)
>
> (2) Recipient: an outlook.com mailbox that I keep for test purposes:
> does not receive from (A) nor from (B), but does receive replies from
> (A) in response to an email that was first sent from outlook.com to (A)
>
> (3) Recipient: a person in a large organization that has switched about
> a year ago to office365 and with whom I have weekly email exchanges:
> receives from (A) and (B), no problem.
>
> If there was at least an indication of rejection, I could try to contact
> my correspondents otherwise.  Instead, Microsoft tells my server that
> everything is OK and then drops the ball, leaving sender and recipient
> in the dark.  In my view, Microsoft's conduct is unacceptable.  If it
> does not deliver an email, it should at least notify the sender SMTP?
>
> Have other SMTP operators been confronted with this?  And what have you
> done about it, if anything?  Is there anything I can do to improve my
> self-operated SMTP server to increase likelihood of delivery through
> Microsoft?  Has anybody been able to engage with Microsoft on the
> subject?  How, through what point of contact, and to what result?
>
> Thanks,
>
> Yuval Levy, JD, MBA, CFA
> Ontario-licensed lawyer
Reply | Threaded
Open this post in threaded view
|

Re: Microsoft silently discarding emails after recepit

Peter Ajamian
On 06/01/18 19:48, [hidden email] wrote:
> I am facing the same issue with hotmail, where Microsoft possibly
> discards all mails with out showing ant error or reason for one of my
> outgoing relay.

It's not the first time I've seen MS accused of dropping mail.  I
suggest you have a look at:
https://mail.live.com/mail/troubleshooting.aspx

Go through all their recommendations and make sure you're following them
correctly.  If you're still having problems I would sign up for SNDS at:
https://postmaster.live.com/snds/index.aspx

...that will allow you to check your IP reputation with Microsoft, if
it's blocked you can contact support and they should remove the block.


Good Luck,


Peter
Reply | Threaded
Open this post in threaded view
|

Re: Microsoft silently discarding emails after recepit

Yuval Levy
TL;DR: IF YOU ARE STILL EXPERIENCING DELIVERABILITY ISSUES, CONTACT
OUTLOOK.COM DELIVERABILITY SUPPORT:
<http://go.microsoft.com/fwlink/?LinkID=614866>

I have contacted Outlook.com Deliverability Support and will report back
to the list if the results from the interaction are of public interest.

On 2018-01-06 02:19 AM, Peter <[hidden email]> wrote:
> It's not the first time I've seen MS accused of dropping mail.

Mine (first post in thread) were not accusations.  They were
corroborated test results.

> I suggest you have a look at:
> https://mail.live.com/mail/troubleshooting.aspx
> [...]
> If you're still having problems I would sign up for SNDS at:
> https://postmaster.live.com/snds/index.aspx
>
> ...that will allow you to check your IP reputation with Microsoft, if
> it's blocked you can contact support and they should remove the block.

Thank you for the reminder.  I was already registered for SNDS just did
not remember.  It is not exactly a popular destination.  Everything
normal there.  I used the Troubleshooting page you linked and worked
through it step by step, to no avail.  All things were already
implemented properly in the past.  One thing I unearthed that was useful:

IF YOU ARE STILL EXPERIENCING DELIVERABILITY ISSUES, CONTACT OUTLOOK.COM
DELIVERABILITY SUPPORT: <http://go.microsoft.com/fwlink/?LinkID=614866>

Things that I have done before contacting outlook.com deliverability
support:

(A) on my sender domain:

* verified that my sender domain has correct SPF set-up
<http://www.kitterman.com/spf/validate.html>
* verified that my sender domain has correct DKIM set-up
<https://protodave.com/tools/dkim-key-checker/>

(B) on my SMTP server:

* verified for configuration changes
* verified that its IP address has a correct PTR record
<https://mxtoolbox.com/ReverseLookup.aspx>
* verified that its IP address is not blacklisted at Spamhaus
<https://www.spamhaus.org/lookup/>
* verified normal IP status at Microsoft's Smart Network Data Services
(SNDS) <https://postmaster.live.com/snds/index.aspx>
* verified that the complaint feed for my server's IP is working
properly and that there is no complaint pending in Microsoft's Junk
Email Reporting Program <https://postmaster.live.com/snds/JMRP.aspx>

(C) test-sent to a non-Microsoft operated domain (e.g. Gmail) and looked
at the source of the received message for

* DKIM-Signature header (was added by my SMTP server as expected)
* Authentication-Results header (return dkim=pass, spf=pass, dmarc=pass)

Things that were recommended on
<https://mail.live.com/mail/troubleshooting.aspx> and were irrelevant:

* Are you sending mail from a new IP?  not applicable
* Are you running Anti-Virus software?  Irrelevant after Outlook
returned a 250 confirming receipt
* Try connecting to mail.hotmail.com via port 25:  Irrelevant after
Outlook returned a 250 confirming receipt, nevertheless, just for the fun:
    nslookup -q=mx hotmail.com
    telnet hotmail-com.olc.protection.outlook.com 25
* Are you advertising yourself as a non-routable IP?  not applicable

Other generally interesting information linked or provided by Microsoft
that is more relevant to setting up proper SMTP operation than to
troubleshooting (and in fairness to Microsoft, it was in their general
policies and guidelines <https://mail.live.com/mail/policies.aspx>
 and not in the troubleshooting section, but still between me and the
link to contact the Outlook.com delverability support):

General Microsoft Policies (if you are a decent netizens, nothing to
worry about)
* Microsoft's Service Agreement
<http://go.microsoft.com/fwlink/?LinkID=530144>
* Microsoft's Anti-Spam Policy
<https://support.office.com/en-us/article/Microsoft-Anti-Spam-Policy-e4506f97-694f-49bc-8231-cac4369afcb8>

Governmental Regulations: (if you are a decent citizen, nothing to worry
about.  Check variation for your jurisdiction (Microsoft is US-centric))
* CAN-SPAM Act
https://www.ftc.gov/tips-advice/business-center/guidance/can-spam-act-compliance-guide-business
* Email Marketers Must Honor "Unsubscribe" Claims
<http://www.lawpublish.com/ftc-emai-marketers-unsubscribe-claims.html>

Technical Guidelines
* Comply with Anti-spam Recommendations for SMTP MTAs (BCP 30)
<http://www.ietf.org/rfc/rfc2505.txt>
* Comply with SMTP Service Extension for Command Pipelining (STD60)
<http://www.ietf.org/rfc/rfc2920.txt>
* Sender is expected to comply with IETF standards, including RFC 2821
<http://www.ietf.org/rfc/rfc2821.txt> and RFC 2822
<http://www.ietf.org/rfc/rfc2822.txt>
* Do not attempt to retransmit same message to same recipient after
receiving an SMTP error response between 500 and 599 (permanent
non-delivery response)
* Do not open more than 500 simultaneous connections to Outlook.com
without making prior arrangements
* Do not transmit messages through insecure email relay or proxy server
* Unsubscribe mechanism to be clearly documented and easy to find and use
* Connections from dynamic IP space may not be accepted
* Email servers must have valid reverse DNS record
* Sender must not use namespace mining technique against Outlook.com
Authentication
* SPF <http://www.ietf.org/rfc/rfc4408.txt>
* DKIM <http://www.ietf.org/rfc/rfc4871.txt>
Reputation Management
* Junk Email Reporting Program (JMRP)
<https://postmaster.live.com/snds/JMRP.aspx>
* Smart Network Data Services (SNDS)
<https://postmaster.live.com/snds/index.aspx>

All of the above I can checkmark in my setup, so I really can't
determine what went wrong.  I recorded all this information in this
public message, hoping that it will become searchable / easily findable
by others in need.  I placed my request with the Outlook.com
deliverability support and am waiting for an answer.  Will report back
if there is anything of public interest.

Thanks,

Yuval Levy, JD, MBA, CFA
Ontario-licensed lawyer
Reply | Threaded
Open this post in threaded view
|

Re: Microsoft silently discarding emails after recepit

stroller-6

> On 6 Jan 2018, at 22:42, Yuval Levy <[hidden email]> wrote:
> On 2018-01-06 02:19 AM, Peter <[hidden email]> wrote:
>> It's not the first time I've seen MS accused of dropping mail.
>
> Mine (first post in thread) were not accusations.  They were
> corroborated test results.

In case English isn't your first language:

• To you, they're corroborated test results.
• To everyone else, they're accusations or allegations.
• To me, they're _very credible_ accusations
• We can't corroborate that your systems are treated this way, because we don't have access to your systems.

Hope this is clear and helpful,

Stroller.

Reply | Threaded
Open this post in threaded view
|

Re: Microsoft silently discarding emails after recepit

Yuval Levy
In reply to this post by Yuval Levy
On 2018-01-06 05:42 PM, Yuval Levy wrote:
> I have contacted Outlook.com Deliverability Support and will report back
> to the list if the results from the interaction are of public interest.

I am still digesting the response received.  In essence, they say that
they "have reviewed [my] IP(s) (XXX.XXX.XXX.XXX) and determined that
messages are being filtered based on the recommendations of the
SmartScreen® Filter.

Email filtering is based on many factors, but primarily it's due to
mail content and recipient interaction with that mail.  Because of the
proprietary nature of SmartScreen® and because SmartScreen® Filter
technology is always adapting and learning more about what is and isn't
unwanted mail, it is not possible for [them] to offer specific advice
about improving your mail content. However, in general SmartScreen®
Filter evaluates specific words or characteristics from each e-mail
message and weights them, based on their likelihood to indicate that a
message is unwanted or legitimate mail."

The above blurb is followed by some advice, most of it in my view
irrelevant or counter-productive since directed at mass-mailed
advertising, and resulting in emails that are in my view more spammy,
not less:
* brand your mail
* highlight opt in
* ensure that your email lists are up to date

Is it possible that Microsoft discriminates against my emails because I
do not use HTML format?  That's an *allegation* I am willing to make at
the right place and time.

Is it possible that Microsoft discriminates against my emails because
they are sent using FLOSS client software and O/S?  That's another
*allegation* I am willing to make at the right place and time.

Is it possible that Microsoft discriminates against my emails because
they show that the IP address from which the mail client sent the
message to the sending MTA is dynamic/residential but there is no trace
of that IP address in consumer databases because I adamantly block any
attempt to track web behaviour at that IP address?  Another *allegation*
that I am willing to make at the right place and time.

Is it possible that Microsoft discriminates against my emails because I
am trying to run a law practice free from Microsoft software and I have
been advocating vocally for access to legal services through software
that works on more than one platform?  Conspiracy theory.  I will not go
that far.

Bottom line, I think the problem is more ethical than technical.  There
is nothing wrong with how my Postfix server and mail client are
configured, but Microsoft decides that it does not like some nebulous
characteristics of my emails on a message by message basis and
(a) substitutes its algorithmic judgment for the judgment of the
recipient as to what is or is not desirable email;
(b) destroys such message without warning or notice to the recipient;
(c) deceives the sender with the misleading statement to the sender's
server that the message has been received and is queued for delivery
(250) when in fact it has been zapped.

I am still thinking how to react.  For now I will advise my clients that
I cannot communicate by email if they are using Microsoft services.

Yuval Levy, JD, MBA, CFA
Ontario-licensed lawyer


Reply | Threaded
Open this post in threaded view
|

Re: Microsoft silently discarding emails after recepit

Viktor Dukhovni


> On Jan 7, 2018, at 12:11 AM, Yuval Levy <[hidden email]> wrote:
>
> Bottom line, I think the problem is more ethical than technical.

Unlikely.  Far more likely your problem does not rise to the level
of negative impact for Microsoft's customer base at large for them
to attempt to tune their filter to solve it.

Microsoft is too big to have a specific agenda against your email,
almost everything they do is done at scale.

Perhaps your IP address is part of a /20 or similar block in which
your "network neighbours" send enough spam to tarnish its reputation.

Generally, the simplest solution is for the correspondents to add
your email address to their contact lists, send you some email, ...
and thereby indicate that your email is wanted by them.

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Microsoft silently discarding emails after recepit

Mike Guelfi
Our alternative was always to just set relays for "poorly behaved" domains to go through the ISP email servers. It was slower but more reliable since the ISP had an artificially inflated reputation and more time to complain when it's email was.blocked.
Reply | Threaded
Open this post in threaded view
|

Re: Microsoft silently discarding emails after recepit

Dominic Raferd
On 07/01/2018 05:11, Yuval Levy wrote:
> On 2018-01-06 05:42 PM, Yuval Levy wrote:
> I am still digesting the response received. In essence, they say that
> they "have reviewed [my] IP(s) (XXX.XXX.XXX.XXX) and determined that
> messages are being filtered based on the recommendations of the
> SmartScreen® Filter.
On 07/01/2018 06:29, Mike Guelfi wrote:
> Our alternative was always to just set relays for "poorly behaved"
> domains to go through the ISP email servers. It was slower but more
> reliable since the ISP had an artificially inflated reputation and
> more time to complain when it's email was.blocked.

Example (using sendgrid for relaying):

/etc/postfix/main.cf:
...
transport_maps = hash:/etc/postfix/transport
...

/etc/postfix/transport:
onedrive.com smtp:smtp.sendgrid.com
onedrive.co.uk smtp:smtp.sendgrid.com
hotmail.com smtp:smtp.sendgrid.com
hotmail.co.uk smtp:smtp.sendgrid.com
microsoft.com smtp:smtp.sendgrid.com
microsoft.co.uk smtp:smtp.sendgrid.com
live.co.uk smtp:smtp.sendgrid.com
live.com smtp:smtp.sendgrid.com
outlook.com smtp:smtp.sendgrid.com
msn.com smtp:smtp.sendgrid.com

Otherwise, the formatting of your DKIM record in DNS seems weird (try:
dig +short 201605sfinacom._domainkey.sfina.com TXT); even if technically
valid the intermediate quotes may be influencing 'SmartScreen'.
Reply | Threaded
Open this post in threaded view
|

Re: Microsoft silently discarding emails after recepit

Peter Ajamian
In reply to this post by Yuval Levy
On 07/01/18 11:42, Yuval Levy wrote:
> On 2018-01-06 02:19 AM, Peter <[hidden email]> wrote:
>> It's not the first time I've seen MS accused of dropping mail.
>
> Mine (first post in thread) were not accusations.  They were
> corroborated test results.

I understand that being a lawyer the word "accused" probably stands out
with a specific legal meaning for you.  I did not intend that meaning, I
simply meant that your complaint is not the first one I've seen.

> Things that were recommended on
> <https://mail.live.com/mail/troubleshooting.aspx> and were irrelevant:
>
> * Are you sending mail from a new IP?  not applicable

How is this not applicable?

> * Are you running Anti-Virus software?  Irrelevant after Outlook
> returned a 250 confirming receipt
> * Try connecting to mail.hotmail.com via port 25:  Irrelevant after
> Outlook returned a 250 confirming receipt, nevertheless, just for the fun:
>     nslookup -q=mx hotmail.com
>     telnet hotmail-com.olc.protection.outlook.com 25
> * Are you advertising yourself as a non-routable IP?  not applicable

Generally speaking, even if you feel or even know for certain that
certain things they are asking for are not relevant or not applicable,
it still pays to do them before asking for support.  The reason here is
simply that you can tell them that you jumped through *all* of their
hoops already so you don't waste time when they insist that you do so
before following up further.

Again, I wish you good luck,


Peter
Reply | Threaded
Open this post in threaded view
|

Re: Microsoft silently discarding emails after recepit

Peter Ajamian
In reply to this post by Yuval Levy
On 07/01/18 18:11, Yuval Levy wrote:
> I am still digesting the response received.  In essence, they say that
> they "have reviewed [my] IP(s) (XXX.XXX.XXX.XXX) and determined that
> messages are being filtered based on the recommendations of the
> SmartScreen® Filter.

You might be interested in this link, then:
https://mail.live.com/mail/junkemail.aspx

> Email filtering is based on many factors, but primarily it's due to
> mail content and recipient interaction with that mail.  Because of the
> proprietary nature of SmartScreen® and because SmartScreen® Filter
> technology is always adapting and learning more about what is and isn't
> unwanted mail, it is not possible for [them] to offer specific advice
> about improving your mail content. However, in general SmartScreen®
> Filter evaluates specific words or characteristics from each e-mail
> message and weights them, based on their likelihood to indicate that a
> message is unwanted or legitimate mail."

So to put it simply, they're basically saying that their black box
thinks that your IP(s) are sending SPAM.

I can speculate that this might be a case of your emails somewhat
resembling emails they have seen from scammers that claim to be legal
representatives in order to further some scam or another.  This is only
a guess though, I could be completely wrong here as I really don't know
anything more about Microsoft's black box than you do.

> Is it possible that Microsoft discriminates against my emails because I
> do not use HTML format?  That's an *allegation* I am willing to make at
> the right place and time.

Highly unlikely.  There is absolutely nothing wrong with sending plain
text emails, I do so myself and don't have problems with MS receiving them.

> Is it possible that Microsoft discriminates against my emails because
> they are sent using FLOSS client software and O/S?  That's another
> *allegation* I am willing to make at the right place and time.

Also highly unlikely, I do the same and don't have issues.  Also it's
all about following the SMTP protocol correctly, which has nothing to do
with whether the software used is FLOSS or proprietary or anything
in-between.

> Is it possible that Microsoft discriminates against my emails because
> they show that the IP address from which the mail client sent the
> message to the sending MTA is dynamic/residential but there is no trace
> of that IP address in consumer databases because I adamantly block any
> attempt to track web behaviour at that IP address?  Another *allegation*
> that I am willing to make at the right place and time.

This shouldn't be the case, but if your submission server resides on a
dynamic IP address then this could very well be the case.  I say
"shouldn't", though, because there have been known cases of anti-spam
appliances in the past that do deep inspection of Received headers and
compare IP addresses found in them against policy blacklists.  These
types of blacklists are designed only to be used against the IP address
of the connecting server, not IPs found in headers.  That said, I don't
think that Microsoft is doing that.

> Is it possible that Microsoft discriminates against my emails because I
> am trying to run a law practice free from Microsoft software and I have
> been advocating vocally for access to legal services through software
> that works on more than one platform?  Conspiracy theory.  I will not go
> that far.

As others have already pointed out, I imagine that you're not big enough
for Microsoft to even care.  I really would not put any malicious intent
here.  There is an old saying, "Never attribute to malice what can
adequately be explained by incompetence."  In this case I think it's
just that you've tripped some stupid anti-spam measure in their black
box and it now thinks that you're a spammer and unfortunately for you
you're too small for Microsoft to care at all about you so they've given
you a cookie-cutter response and file the complaint away in their
virtual rubbish bin.

> Bottom line, I think the problem is more ethical than technical.

I certainly think it's technical on their part, but if by ethical you
mean that Microsoft just doesn't care enough about you to want to solve
your problem then you're probably right.

>  There
> is nothing wrong with how my Postfix server and mail client are
> configured, but Microsoft decides that it does not like some nebulous
> characteristics of my emails on a message by message basis and
> (a) substitutes its algorithmic judgment for the judgment of the
> recipient as to what is or is not desirable email;
> (b) destroys such message without warning or notice to the recipient;
> (c) deceives the sender with the misleading statement to the sender's
> server that the message has been received and is queued for delivery
> (250) when in fact it has been zapped.

I certainly agree with you on these points, but again I don't read
anything malicious against you personally.

> I am still thinking how to react.  For now I will advise my clients that
> I cannot communicate by email if they are using Microsoft services.

I would suggest, as others have, that if you cannot resolve this
directly then you use a relayhost for messages that go out to Microsoft
clients, then you should at least be able to get your mail through.


Good Luck,


Peter
Reply | Threaded
Open this post in threaded view
|

Re: Microsoft silently discarding emails after recepit

Stephen Satchell
In reply to this post by Yuval Levy
On 01/06/2018 09:11 PM, Yuval Levy wrote:
> On 2018-01-06 05:42 PM, Yuval Levy wrote:
>> I have contacted Outlook.com Deliverability Support and will report back
>> to the list if the results from the interaction are of public interest.

After reading all the responses, and reading the reference links about
Microsoft Smartscreen, I have a really stupid question:

Have you considered encrypting your e-mail traffic?

You are a lawyer.  The contents of your mail could be considered
sensitive, especially if exposure of the mail could materially affect
the legal status of your clients.  By encrypting your mail, you
instantly remove the idea that the message you are sending is "bulk" in
any way, because the methods used to encrypt email involve key-pairs
specific to you and your client.  Using public keys to encrypt, only the
secret private key can be used to decrypt -- and Microsoft wouldn't have
access to the secret keys.

But that is a discussion for a different mailing list.  PostFix, to the
best of my knowledge, doesn't have a way to encrypt content "on the
fly".  I use EnigMail in Thunderbird, which is a shell for GPG (Gnu
Privacy Guard), with those people whose messages need to be kept close
to the vest.

Hmmm....need to experiment.  Could I write a milter for PostFix that
would (1) detect the message body is in plaintext, (2) the recipient
address has a public key listed in the key servers, and (3) encrypt the
body of the message.
Reply | Threaded
Open this post in threaded view
|

Re: Microsoft silently discarding emails after recepit

Yuval Levy
In reply to this post by Viktor Dukhovni
On 2018-01-07 01:07 AM, Viktor Dukhovni wrote:
> Perhaps your IP address is part of a /20 or similar block in which
> your "network neighbours" send enough spam to tarnish its reputation.

I have just sent to the list the complete thread with Microsoft until
now, hopefully it properly replied to this thread.

My understanding of Microsoft's reply is that there is nothing wrong
with the server / IP / reputation.  Their reply point to a content
filter at work.


> Generally, the simplest solution is for the correspondents to add
> your email address to their contact lists, send you some email, ...
> and thereby indicate that your email is wanted by them.

If I could exercise that level of control on my clients, I would direct
them to a non-Microsoft service.  There are a bunch of other problems
with contact lists on free email services.  Bottom line is that it is
not a practicable solution in my circumstances, and further discussion
would detract from the main issue which is "silently discarded emails,"
I behavior that in my view is plain wrong and threatens the usefulness
of email more than a few false positive spam messages.

Yuv
Reply | Threaded
Open this post in threaded view
|

Re: Microsoft silently discarding emails after recepit

Viktor Dukhovni


> On Jan 7, 2018, at 2:15 PM, Yuval Levy <[hidden email]> wrote:
>
> My understanding of Microsoft's reply is that there is nothing wrong
> with the server / IP / reputation.  Their reply point to a content
> filter at work.

There are multiple factors that go into black-box decisions by spam
detection engines.  Though your IP address may ok, the IP block it is
in may be less so.

I agree that silent discard is rather drastic, and ideally false-positive
messages other than malware should end up in the "Junk" folder for the user
to find as needed.

It seems there's not much you can do.  If you have other means to reach
your clients ask them to get a mailbox at a different provider or add
you to their contact list.  If they are unwilling to do that, perhaps
they don't especially want to receive your email, in which case, the
anti-spam filter is doing what the user wants... :-(

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: Microsoft silently discarding emails after recepit

Richard Damon
On 1/7/18 3:03 PM, Viktor Dukhovni wrote:

>
>> On Jan 7, 2018, at 2:15 PM, Yuval Levy <[hidden email]> wrote:
>>
>> My understanding of Microsoft's reply is that there is nothing wrong
>> with the server / IP / reputation.  Their reply point to a content
>> filter at work.
> There are multiple factors that go into black-box decisions by spam
> detection engines.  Though your IP address may ok, the IP block it is
> in may be less so.
>
> I agree that silent discard is rather drastic, and ideally false-positive
> messages other than malware should end up in the "Junk" folder for the user
> to find as needed.
>
> It seems there's not much you can do.  If you have other means to reach
> your clients ask them to get a mailbox at a different provider or add
> you to their contact list.  If they are unwilling to do that, perhaps
> they don't especially want to receive your email, in which case, the
> anti-spam filter is doing what the user wants... :-(
>
Fundamentally, this is a basic issue with ALL the free Email services.
The service is given to you for free, normally so they can learn more
about you, and show you advertisements (so they can earn the income
needed to pay for the service), and generally you get what you pay for.

Their user agreements basically disavow any implication that the service
will be reliable or fit for use. The silent dropping of messages is
basically expected behavior.

--
Richard Damon

Reply | Threaded
Open this post in threaded view
|

Re: Microsoft silently discarding emails after recepit

Yuval Levy
In reply to this post by Dominic Raferd
Thank you for the useful and practical advice.
On 2018-01-07 03:57 AM, Dominic Raferd wrote:

> On 07/01/2018 06:29, Mike Guelfi wrote:
>> Our alternative was always to just set relays for "poorly behaved"
>> domains
>
> Example (using sendgrid for relaying):
>
> /etc/postfix/main.cf:
> ...
> transport_maps = hash:/etc/postfix/transport
> ...
>
> /etc/postfix/transport:
> onedrive.com smtp:smtp.sendgrid.com

I have two main issues with the specific example above.

(1) It is a significant degradation to me if the MTA that receives the
email from my MTA is not operated by the same entity that operates the MDA.

(2) I have looked at sendgrid.com and to me it looks like a service to
increase the reach of unwanted ads.  I want less ads, not more.  The
mere sentence "Email Marketing Campaign" sends negative shivers down my
spine, and I find it objectionable to reward a third party that improves
the delivery of spam with additional revenue.  In my view, the only
legit payment flow in addition to paying for ISP would be some sort of
stamp whose value flows automatically and directly from the writer's
pocket to the reader's pocket with no reward to any intermediary or
other third party.  The intermediary should stay neutral.  Spam is in
the eyes of the reader and if an entity sends you a message that wastes
your time, you should have a right to be compensated for the pollution.
Ideally the compensation should be designed to dissuade spam.  Different
subject.


> Otherwise, the formatting of your DKIM record in DNS seems weird (try:
> dig +short 201605sfinacom._domainkey.sfina.com TXT); even if technically
> valid the intermediate quotes may be influencing 'SmartScreen'.

The intermediate quotes are a result of key length.  When I use 2048 bit
length, there are intermediate quotes.  When I use 1024 bit length,
there are not:

SELECTOR=<YYYYMM>
DOMAIN=<domain.com>
opendkim-genkey -b 2048 -r -s $SELECTOR$DOMAIN_2 -d $DOMAIN
opendkim-genkey -b 1024 -r -s $SELECTOR$DOMAIN_1 -d $DOMAIN

When I implemented DKIM, I have merely adapted the tutorial at
<https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-dkim-with-postfix-on-debian-wheezy>
to my needs, checked the DKIM Key at
<https://protodave.com/tools/dkim-key-checker/> and verified that my
set-up is full operational with an email to
[hidden email] and one to
<https://www.mail-tester.com/>.  Never had any problem with this.

Since it worked, so I did not bother digging deeper into the detail.  I
am familiar enough with encryption to know the bit length trade-off with
the longer key imposing a computing cost on the receiving end in
exchange for longevity.

What is the experience of the experts on this list?  Do server ignore
DKIM when its key length is too long, and how long is the ideal key
length?  I am interested to learn more.  My choice of the longest keys
is because I am a self-supporting one man show and the further out in
the future I can postpone the regeneration of encryption keys, the less
time I need to spend on this.  My next planned reset of my
infrastructure is sometimes in the Spring/Summer, on Ubuntu 18.04LTS,
and I hope that the set up will hum along for four years with minimal
intervention.

Yuv




Reply | Threaded
Open this post in threaded view
|

Re: Microsoft silently discarding emails after recepit

Yuval Levy
In reply to this post by Peter Ajamian
On 2018-01-07 04:08 AM, Peter wrote:
> simply meant that your complaint is not the first one I've seen.

OK ;-)


>> * Are you sending mail from a new IP?  not applicable
>
> How is this not applicable?

I've operated that IP for the past four years, it is older than my legal
career.  I realize that the comments in my mail are specific to my
situation and I did not made it clear enough.  Trying to mix a general
bullet-point roadmap with the specific situation was a bad idea.


>> * Are you running Anti-Virus software?  Irrelevant after Outlook
>> returned a 250 confirming receipt
>> * Try connecting to mail.hotmail.com via port 25:  Irrelevant after
>> Outlook returned a 250 confirming receipt, nevertheless, just for the fun:
>>     nslookup -q=mx hotmail.com
>>     telnet hotmail-com.olc.protection.outlook.com 25
>> * Are you advertising yourself as a non-routable IP?  not applicable
>
> Generally speaking, even if you feel or even know for certain that
> certain things they are asking for are not relevant or not applicable,
> it still pays to do them before asking for support.  The reason here is
> simply that you can tell them that you jumped through *all* of their
> hoops already so you don't waste time when they insist that you do so
> before following up further.

Completely agree, which is why I addressed them in my checklist instead
of being silent about them and making it look as if I ignored those points.

Yuv
Reply | Threaded
Open this post in threaded view
|

Re: Microsoft silently discarding emails after recepit

D'Arcy Cain
In reply to this post by Yuval Levy
On 01/07/2018 01:15 PM, Yuval Levy wrote:
> would detract from the main issue which is "silently discarded emails,"
> I behavior that in my view is plain wrong and threatens the usefulness
> of email more than a few false positive spam messages.

Absolutely.  There are only two things that an MTA should do with email,
deliver it or bounce it.  Silently dropping is plain wrong.

--
D'Arcy J.M. Cain
System Administrator, Vex.Net
http://www.Vex.Net/ IM:[hidden email]
VoIP: sip:[hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Microsoft silently discarding emails after recepit

Viktor Dukhovni


> On Jan 7, 2018, at 6:09 PM, D'Arcy Cain <[hidden email]> wrote:
>
> Absolutely.  There are only two things that an MTA should do with email,
> deliver it or bounce it.  Silently dropping is plain wrong.

There's a reasonable exception for malware.  Detection of malware has
a much lower FP rate than detection of spam.  There's little benefit
and much risk in delegating the decision to read or not read messages
that have malicious content.  The only plausible exception is ".doc"
or similar resumes with viruses removed, if one still wants to work
with candidates whose machines are infected...  If the virus scan
engine has sufficiently fine-grained classification one might then
permit messages where document macros are sanitized, and the removed
macros are not of the email-worm variety.  Don't know whether any
AV software makes this possible.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: Microsoft silently discarding emails after recepit

Stephen Satchell
In reply to this post by D'Arcy Cain
On 01/07/2018 03:09 PM, D'Arcy Cain wrote:
> On 01/07/2018 01:15 PM, Yuval Levy wrote:
>> would detract from the main issue which is "silently discarded emails,"
>> I behavior that in my view is plain wrong and threatens the usefulness
>> of email more than a few false positive spam messages.
>
> Absolutely.  There are only two things that an MTA should do with email,
> deliver it or bounce it.  Silently dropping is plain wrong.

But the best time to bounce the mail is during initial delivery.
Microsoft makes it clear that their Smart-whatever is done *after* the
SMTP server has accepted the mail.  Rejecting the mail afterwards has
the risk of sending the bounce not to the sender, but to another party
whose name just happened to be in the From: field.

Most of the spam I've received over the years doesn't have a live
account as specified in the From: field.  So the problem is, where to
send the bounce notification if you already said "I got it"?

You guessed:  you don't.  You drop the mail, and that's that.  No
side-channel spaming.

When I smart-hosted a bunch of Plesk and CPanel systems with edge
PostFix servers, I had to be very careful not to run into this very same
situation with quotas.  It was tedious and complicated and ugly, but I
made it work.  I also had those same servers accept mail from the Web
boxes and did anti-spam, dropping "bad" e-mails after sending a note to
my admin logs about the dirty delivery.

(I was able to find the small number of spammers in a community of
thousands of accounts this way, and closed them due to violations of the
terms of service.)
12