When a 554 acts like a 471?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

When a 554 acts like a 471?

Bob Proulx
I have a case that is odd to me and I can't figure it out.  Hopefully
someone here will be able to set me straight.  This is on a friend's
system that I am helping to maintain.

My friend somewhat out of the blue decided to start sending mail from
a rented VM server.  I hadn't expected and don't think it is really
set up as it should for it.  But people do what people do.  It is not
set up with DKIM.  "inet_interfaces = loopback-only" so it cannot
receive mail and therefore cannot possibly be relaying spam.  This was
a clean IP as far as I could tell and my friend has used it for the
last few years.  Therefore I know that it has not generated any email
at all other than what my friend has recently decided to send out
personally.

  root@ergo:~$ grep 4E7541E092 /var/log/mail.log

  Dec 19 13:11:55 ergo postfix/qmgr[2592]: 4E7541E092: from=<[[redacted]]>, size=4861, nrcpt=2 (queue active)
  Dec 19 13:11:55 ergo postfix/smtp[7858]: 4E7541E092: to=<[[redacted]]@fortboise.org>, relay=mx.fortboise.org.cust.a.hostedemail.com[216.40.42.4]:25, delay=315407, delays=315407/0.02/0.14/0, dsn=4.7.1, status=deferred (host mx.fortboise.org.cust.a.hostedemail.com[216.40.42.4] refused to talk to me: 554 5.7.1 Service unavailable; Client host [93.184.216.34] blocked using urbl.hostedemail.com; Your IP has been manually blacklisted)

  Dec 19 14:21:55 ergo postfix/qmgr[2592]: 4E7541E092: from=<[[redacted]]>, size=4861, nrcpt=2 (queue active)
  Dec 19 14:21:55 ergo postfix/smtp[19888]: 4E7541E092: to=<[[redacted]]@fortboise.org>, relay=mx.fortboise.org.cust.a.hostedemail.com[216.40.42.4]:25, delay=319607, delays=319607/0.02/0.18/0, dsn=4.7.1, status=deferred (host mx.fortboise.org.cust.a.hostedemail.com[216.40.42.4] refused to talk to me: 554 5.7.1 Service unavailable; Client host [93.184.216.34] blocked using urbl.hostedemail.com; Your IP has been manually blacklisted)

And the above repeats.  I made some obvious redactions and
93.184.216.34 is actually example.com but you can understand me
keeping my friend's information out of my email here.  The IP listed
was the IP of my friend's VM originating the mail.

There are currently 15 requests in the queue.  Which all appear to be
personal correspondence that my friend typed in.  Prolific!  They
eventually timeout after, I think, bounce_queue_lifetime as expected.
Here is a sample of the bounce message generated.  Which is why I am
involved trying to help.

  <[[redacted]]@fortboise.org>: host mx.fortboise.org.cust.a.hostedemail.com[216.40.42.4]
      refused to talk to me: 554 5.7.1 Service unavailable; Client host
      [93.184.216.34] blocked using urbl.hostedemail.com; Your IP has been
      manually blacklisted

But this confuses me.  It appears to me that the message was rejected
at SMTP time with a 554 code.  Therefore shouldn't that generate a
bounce message immediately?  Why is dsn=4.7.1 being logged there?

Was the actual SMTP rejecting a 554?  Or was it a 471?  I feel it must
have been a 471 because it is retrying instead of immediately
bouncing.  Yet here it is saying it was a 554.  In which case why
isn't it bouncing immediately?

I will include some hopefully useful information below.

Thanks!
Bob

root@ergo:~# postconf -n
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
append_dot_mydomain = no
biff = no
compatibility_level = 2
inet_interfaces = loopback-only
inet_protocols = ipv4
local_header_rewrite_clients = permit_mynetworks, permit_sasl_authenticated
mailbox_command = procmail -a "$EXTENSION"
mailbox_size_limit = 0
masquerade_classes = envelope_sender, header_sender
masquerade_domains = [[redacted]]
masquerade_exceptions = root
mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain
myhostname = ergo.[[redacted]]
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
readme_directory = no
recipient_delimiter = +
relayhost =
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
smtpd_tls_cert_file = /etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file = /etc/ssl/private/ssl-cert-snakeoil.key
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtpd_use_tls = yes

This is on the Debian 9 Stretch version of Postfix for which I will
try to avoid the gasp of age from the version number as I don't think
that is the issue here. :-)
Reply | Threaded
Open this post in threaded view
|

Re: When a 554 acts like a 471?

Noel Jones-2
On 12/19/2019 3:54 PM, Bob Proulx wrote:

> I have a case that is odd to me and I can't figure it out.  Hopefully
> someone here will be able to set me straight.  This is on a friend's
> system that I am helping to maintain.
>
> My friend somewhat out of the blue decided to start sending mail from
> a rented VM server.  I hadn't expected and don't think it is really
> set up as it should for it.  But people do what people do.  It is not
> set up with DKIM.  "inet_interfaces = loopback-only" so it cannot
> receive mail and therefore cannot possibly be relaying spam.  This was
> a clean IP as far as I could tell and my friend has used it for the
> last few years.  Therefore I know that it has not generated any email
> at all other than what my friend has recently decided to send out
> personally.
>
>    root@ergo:~$ grep 4E7541E092 /var/log/mail.log
>
>    Dec 19 13:11:55 ergo postfix/qmgr[2592]: 4E7541E092: from=<[[redacted]]>, size=4861, nrcpt=2 (queue active)
>    Dec 19 13:11:55 ergo postfix/smtp[7858]: 4E7541E092: to=<[[redacted]]@fortboise.org>, relay=mx.fortboise.org.cust.a.hostedemail.com[216.40.42.4]:25, delay=315407, delays=315407/0.02/0.14/0, dsn=4.7.1, status=deferred (host mx.fortboise.org.cust.a.hostedemail.com[216.40.42.4] refused to talk to me: 554 5.7.1 Service unavailable; Client host [93.184.216.34] blocked using urbl.hostedemail.com; Your IP has been manually blacklisted)
>
>    Dec 19 14:21:55 ergo postfix/qmgr[2592]: 4E7541E092: from=<[[redacted]]>, size=4861, nrcpt=2 (queue active)
>    Dec 19 14:21:55 ergo postfix/smtp[19888]: 4E7541E092: to=<[[redacted]]@fortboise.org>, relay=mx.fortboise.org.cust.a.hostedemail.com[216.40.42.4]:25, delay=319607, delays=319607/0.02/0.18/0, dsn=4.7.1, status=deferred (host mx.fortboise.org.cust.a.hostedemail.com[216.40.42.4] refused to talk to me: 554 5.7.1 Service unavailable; Client host [93.184.216.34] blocked using urbl.hostedemail.com; Your IP has been manually blacklisted)
>
> And the above repeats.  I made some obvious redactions and
> 93.184.216.34 is actually example.com but you can understand me
> keeping my friend's information out of my email here.  The IP listed
> was the IP of my friend's VM originating the mail.
>
> There are currently 15 requests in the queue.  Which all appear to be
> personal correspondence that my friend typed in.  Prolific!  They
> eventually timeout after, I think, bounce_queue_lifetime as expected.
> Here is a sample of the bounce message generated.  Which is why I am
> involved trying to help.
>
>    <[[redacted]]@fortboise.org>: host mx.fortboise.org.cust.a.hostedemail.com[216.40.42.4]
>        refused to talk to me: 554 5.7.1 Service unavailable; Client host
>        [93.184.216.34] blocked using urbl.hostedemail.com; Your IP has been
>        manually blacklisted
>
> But this confuses me.  It appears to me that the message was rejected
> at SMTP time with a 554 code.  Therefore shouldn't that generate a
> bounce message immediately?  Why is dsn=4.7.1 being logged there?
>
> Was the actual SMTP rejecting a 554?  Or was it a 471?  I feel it must
> have been a 471 because it is retrying instead of immediately
> bouncing.  Yet here it is saying it was a 554.  In which case why
> isn't it bouncing immediately?
>


The remote server greeted postfix with a 554 code.  By default,
postfix defers mail when the remote server does this. If you would
rather have postfix immediately bounce the mail, see:
http://www.postfix.org/postconf.5.html#smtp_skip_5xx_greeting

As for why it's being rejected, you'll need to contact the relay
postmaster. Looks like the IP has been "manually blacklisted," so
one would presume it will need to be manually whitelisted.




   -- Noel Jones
Reply | Threaded
Open this post in threaded view
|

Re: When a 554 acts like a 471?

Bob Proulx
Noel Jones wrote:
> Bob Proulx wrote:
> > But this confuses me.  It appears to me that the message was rejected
> > at SMTP time with a 554 code.  Therefore shouldn't that generate a
> > bounce message immediately?  Why is dsn=4.7.1 being logged there?
>
> The remote server greeted postfix with a 554 code.  By default, postfix
> defers mail when the remote server does this. If you would rather have
> postfix immediately bounce the mail, see:
> http://www.postfix.org/postconf.5.html#smtp_skip_5xx_greeting

  "By default, the Postfix SMTP client moves on the next mail
  exchanger. Specify "smtp_skip_5xx_greeting = no" if Postfix should
  bounce the mail immediately. Caution: the latter behavior appears to
  contradict RFC 2821."

Hmm...  I read through RFC 2821 and it isn't clear to me why 554
Transaction Failed needs to be interpreted as a temporary failure
needing a retry.  I am sure there is some historical experience of
servers using 554 for temporary conditions for which I am missing that
makes that behavior the correct default behavior.  I thought all of
the 5xx codes were non-temporary and should be treated as hard errors.

But regardless that is exactly the answer I needed!  I am going to set
that for my friend's system as I think that is the best setting.

> As for why it's being rejected, you'll need to contact the relay postmaster.
> Looks like the IP has been "manually blacklisted," so one would presume it
> will need to be manually whitelisted.

Right.  But I have no dillusions about that happening with
hostedemail.com which does not seem to be responsive.  And actually I
would not be surprised if the recipient simply decided to report the
messages as spam as a lazy way of saying don't type at me anymore.

Thank you for the information!  It got me unstuck. :-)

Bob
Reply | Threaded
Open this post in threaded view
|

Re: When a 554 acts like a 471?

Viktor Dukhovni


> On Dec 19, 2019, at 7:00 PM, Bob Proulx <[hidden email]> wrote:
>
>  "By default, the Postfix SMTP client moves on the next mail
>  exchanger. Specify "smtp_skip_5xx_greeting = no" if Postfix should
>  bounce the mail immediately. Caution: the latter behavior appears to
>  contradict RFC 2821."
>
> But regardless that is exactly the answer I needed!  I am going to set
> that for my friend's system as I think that is the best setting.

Actually, it may not be.  You may end up losing mail when you
encounter overloaded servers returning 554.

In a bit of synchronicity, just a day or two ago I was corresponding
on this very question with Wietse, because there's an effort underway
to polish RFC5321 with a goal of making SMTP a "full" Internet Standard
(it only takes ~40 years).  As part of that, I'd like to see a couple
of the murkier bits clarified:

  * The 552 -> 452 work-around, that should no longer (if ever)
    be necessary:

    https://mailarchive.ietf.org/arch/msg/ietf-smtp/fhKAX0Z9frJxUxmZylthJ_LijMM

  * The semantics of 554 greetings:

    https://mailarchive.ietf.org/arch/msg/ietf-smtp/NeHzVodzZX6kJFJ7lQhuP6gOpWA

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: When a 554 acts like a 471?

Bob Proulx
Viktor Dukhovni wrote:

> > Bob Proulx wrote:
> >  "By default, the Postfix SMTP client moves on the next mail
> >  exchanger. Specify "smtp_skip_5xx_greeting = no" if Postfix should
> >  bounce the mail immediately. Caution: the latter behavior appears to
> >  contradict RFC 2821."
> >
> > But regardless that is exactly the answer I needed!  I am going to set
> > that for my friend's system as I think that is the best setting.
>
> Actually, it may not be.  You may end up losing mail when you
> encounter overloaded servers returning 554.

In my friend's case an immediate response to a message being sent just
now would be better feedback.  As opposed to after the bounce timeout
occurs some five days later.  Email is generally so very reliable
these days that when we hear nothing we assume it works.  And also
that it went straight into the recipient's Junk folder.

> In a bit of synchronicity, just a day or two ago I was corresponding
> on this very question with Wietse, because there's an effort underway

What a coincidence!

> to polish RFC5321 with a goal of making SMTP a "full" Internet Standard
> (it only takes ~40 years).  As part of that, I'd like to see a couple
> of the murkier bits clarified:
>
>   * The 552 -> 452 work-around, that should no longer (if ever)
>     be necessary:
>
>     https://mailarchive.ietf.org/arch/msg/ietf-smtp/fhKAX0Z9frJxUxmZylthJ_LijMM
>
>   * The semantics of 554 greetings:
>
>     https://mailarchive.ietf.org/arch/msg/ietf-smtp/NeHzVodzZX6kJFJ7lQhuP6gOpWA

And I see I am not the only one to be confused by the use of a 5xx
code instead of a 4xx code, or the reverse, for this.  The background
in the above references was indeed very interesting.  It helps to
bring me up to speed on the issue.

Forty years!  Well...  We don't want to rush into these things.  After
all this whole electronic mail thing might just be a fad. :-)

Thanks!
Bob