Forwarding received mail through AWS SES

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Forwarding received mail through AWS SES

Yasuhiro KIMURA
Hello,

I use Debian 9 on AWS EC2. If mail is sent directly from EC2 host then
some mail service provider such as Gmail rejects receiving it. So I set up
so that mail is sent through AWS SES with following steps.

1. Obtain SES SMTP credential accoring to following document
   https://docs.aws.amazon.com/ses/latest/DeveloperGuide/smtp-credentials.html
2. Verify domain with SES accoring to following document
   https://docs.aws.amazon.com/ses/latest/DeveloperGuide/verify-domain-procedure.html
3. Move the domain out of SES sandbox accoring to following document
   https://docs.aws.amazon.com/ses/latest/DeveloperGuide/request-production-access.html
4. Set up postfix to following document
   https://docs.aws.amazon.com/ses/latest/DeveloperGuide/postfix.html

And postfix is configured so that 'postconf -n' shows as following.

----------------------------------------------------------------------
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
append_dot_mydomain = yes
biff = no
compatibility_level = 2
home_mailbox = Maildir/
inet_interfaces = all
inet_protocols = all
mailbox_size_limit = 0
milter_default_action = accept
mydestination = $myhostname, $mydomain, localhost.$mydomain, localhost
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
myorigin = /etc/mailname
readme_directory = no
recipient_delimiter = +
relayhost = email-smtp.us-east-1.amazonaws.com:587
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtp_tls_ciphers = high
smtp_tls_loglevel = 1
smtp_tls_mandatory_ciphers = high
smtp_tls_mandatory_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1
smtp_tls_note_starttls_offer = yes
smtp_tls_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1
smtp_tls_security_level = encrypt
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
smtpd_milters = inet:localhost:11332
smtpd_recipient_restrictions = permit_mynetworks reject_unauth_destination check_policy_service inet:127.0.0.1:10023
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
smtpd_sasl_path = private/auth
smtpd_sasl_security_options = noplaintext,noanonymous
smtpd_sasl_tls_security_options = noanonymous
smtpd_sasl_type = dovecot
smtpd_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtpd_tls_cert_file = /etc/letsencrypt/live/examle.org/fullchain.pem
smtpd_tls_ciphers = high
smtpd_tls_key_file = /etc/letsencrypt/live/examle.org/privkey.pem
smtpd_tls_loglevel = 1
smtpd_tls_mandatory_ciphers = high
smtpd_tls_mandatory_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1
smtpd_tls_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1
smtpd_tls_security_level = may
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
----------------------------------------------------------------------

And with this settings I confirmed that mail is successfully delivered
from the EC2 host to destination that mail is rejected if delivered
directly.

But if I try to forward mail sent from outside to this host by using
~/.forward or something else, then SES rejects such mail with
following log messages.

----------------------------------------------------------------------
Jan 19 14:53:32 server postfix/smtpd[19403]: connect from gate.example.com[10.0.0.1]
Jan 19 14:53:32 server postfix/smtpd[19403]: Anonymous TLS connection established from gate.example.com[10.0.0.1]: TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)
Jan 19 14:53:32 server postfix/smtpd[19403]: 629A74F3: client=gate.example.com[10.0.0.1]
Jan 19 14:53:32 server postfix/cleanup[19410]: 629A74F3: message-id=<[hidden email]>
Jan 19 14:53:36 server postfix/qmgr[16757]: 629A74F3: from=<[hidden email]>, size=1495, nrcpt=1 (queue active)
Jan 19 14:53:36 server postfix/smtpd[19403]: disconnect from gate.example.com[10.0.0.1] ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7
Jan 19 14:53:36 server postfix/local[19411]: 629A74F3: to=<[hidden email]>, relay=local, delay=4.1, delays=4.1/0.01/0/0, dsn=2.0.0, status=sent (delivered to maildir)
Jan 19 14:53:36 server postfix/cleanup[19410]: 7FF264F4: message-id=<[hidden email]>
Jan 19 14:53:36 server postfix/qmgr[16757]: 7FF264F4: from=<[hidden email]>, size=2199, nrcpt=2 (queue active)
Jan 19 14:53:36 server postfix/local[19411]: 629A74F3: to=<[hidden email]>, relay=local, delay=4.1, delays=4.1/0.01/0/0.01, dsn=2.0.0, status=sent (forwarded as 7FF264F4)
Jan 19 14:53:36 server postfix/qmgr[16757]: 629A74F3: removed
Jan 19 14:53:38 server postfix/smtp[19412]: Trusted TLS connection established to email-smtp.us-east-1.amazonaws.com[23.23.175.128]:587: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Jan 19 14:53:39 server postfix/smtp[19412]: 7FF264F4: to=<[hidden email]>, orig_to=<[hidden email]>, relay=email-smtp.us-east-1.amazonaws.com[23.23.175.128]:587, delay=2.7, delays=0.01/0.02/1.8/0.85, dsn=5.0.0, status=bounced (host email-smtp.us-east-1.amazonaws.com[23.23.175.128] said: 554 Message rejected: Email address is not verified. The following identities failed the check in region US-EAST-1: [hidden email], example.com mail user <[hidden email]> (in reply to end of DATA command))
Jan 19 14:53:39 server postfix/cleanup[19410]: 5C8474F5: message-id=<[hidden email]>
Jan 19 14:53:39 server postfix/qmgr[16757]: 5C8474F5: from=<>, size=5389, nrcpt=1 (queue active)
Jan 19 14:53:39 server postfix/bounce[19413]: 7FF264F4: sender non-delivery notification: 5C8474F5
Jan 19 14:53:39 server postfix/qmgr[16757]: 7FF264F4: removed
Jan 19 14:53:40 server postfix/smtp[19412]: Trusted TLS connection established to email-smtp.us-east-1.amazonaws.com[54.225.136.195]:587: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Jan 19 14:53:41 server postfix/smtp[19412]: 5C8474F5: to=<[hidden email]>, relay=email-smtp.us-east-1.amazonaws.com[54.225.136.195]:587, delay=1.7, delays=0/0/1.5/0.16, dsn=5.0.0, status=bounced (host email-smtp.us-east-1.amazonaws.com[54.225.136.195] said: 501 Invalid MAIL FROM address provided (in reply to MAIL FROM command))
Jan 19 14:53:41 server postfix/qmgr[16757]: 5C8474F5: removed
----------------------------------------------------------------------

Then how should I configure postfix to forward mail from outside
through SES? Please let me know if someone succeeds it.

Best Regards.

---
Yasuhiro KIMURA
Reply | Threaded
Open this post in threaded view
|

Re: Forwarding received mail through AWS SES

Christos Chatzaras
AWS EC2 IPs may have low reputation to e-mail providers, so is not recommended to send e-mails using these IPs.

Also AWS SES frequently have issues with RBLs. I wouldn't use it if you use reliable delivery. It's good for newsletters because it has low cost compared to other services and when you don't care if some e-mails are not delivered.

My recommendation is to setup a VPS (from a company that has clean network) with multiple IPs if you need to send a lot of messages and use postfix relay with randmap to balance the outgoing messages between the IPs.
Reply | Threaded
Open this post in threaded view
|

Re: Forwarding received mail through AWS SES

Yasuhiro KIMURA
From: Christos Chatzaras <[hidden email]>
Subject: Re: Forwarding received mail through AWS SES
Date: Sat, 19 Jan 2019 12:35:58 +0200

> AWS EC2 IPs may have low reputation to e-mail providers, so is not recommended to send e-mails using these IPs.
>
> Also AWS SES frequently have issues with RBLs. I wouldn't use it if you use reliable delivery. It's good for newsletters because it has low cost compared to other services and when you don't care if some e-mails are not delivered.
>
> My recommendation is to setup a VPS (from a company that has clean network) with multiple IPs if you need to send a lot of messages and use postfix relay with randmap to balance the outgoing messages between the IPs.

Thank you for reply. Then I consider VPS instead of AWS EC2 and SES.

---
Yasuhiro KIMURA
Reply | Threaded
Open this post in threaded view
|

Re: Forwarding received mail through AWS SES

Bill Cole-3
In reply to this post by Yasuhiro KIMURA
On 19 Jan 2019, at 4:03, Yasuhiro KIMURA wrote:

> Hello,
>
> I use Debian 9 on AWS EC2. If mail is sent directly from EC2 host then
> some mail service provider such as Gmail rejects receiving it. So I
> set up
> so that mail is sent through AWS SES with following steps.
>
> 1. Obtain SES SMTP credential accoring to following document
>    https://docs.aws.amazon.com/ses/latest/DeveloperGuide/smtp-credentials.html
> 2. Verify domain with SES accoring to following document
>    https://docs.aws.amazon.com/ses/latest/DeveloperGuide/verify-domain-procedure.html
> 3. Move the domain out of SES sandbox accoring to following document
>    https://docs.aws.amazon.com/ses/latest/DeveloperGuide/request-production-access.html
> 4. Set up postfix to following document
>    https://docs.aws.amazon.com/ses/latest/DeveloperGuide/postfix.html

Notably missing from that list:

"Receiving Email with Amazon SES"
https://docs.aws.amazon.com/ses/latest/DeveloperGuide/receiving-email.html

Which seems critical if you want SES to accept mail for you.

> And postfix is configured so that 'postconf -n' shows as following.

[...]
> And with this settings I confirmed that mail is successfully delivered
> from the EC2 host to destination that mail is rejected if delivered
> directly.
>
> But if I try to forward mail sent from outside to this host by using
> ~/.forward or something else, then SES rejects such mail with
> following log messages.

Yes, which is *probably* because ~/.forward or aliases use 'transparent'
forwarding, using the same SMTP envelope sender as the incoming message
and making no modifications to the message itself except to add a
Received header.

Because it is an Amazon SES machine which is rejecting the forwarded
email, only Amazon can provide a definitive answer to your query of how
to make this work. You may need to modify the SMTP envelope sender to an
address in your domain, you may need to modify headers, you may need to
set up DKIM signing, or do something else Amazon-specific.


--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Reply | Threaded
Open this post in threaded view
|

Re: Forwarding received mail through AWS SES

Durga Prasad Malyala
In reply to this post by Yasuhiro KIMURA

On Sat, Jan 19, 2019, 23:26 Yasuhiro KIMURA <[hidden email] wrote:
From: Christos Chatzaras <[hidden email]>
Subject: Re: Forwarding received mail through AWS SES
Date: Sat, 19 Jan 2019 12:35:58 +0200

> AWS EC2 IPs may have low reputation to e-mail providers, so is not recommended to send e-mails using these IPs.
>
> Also AWS SES frequently have issues with RBLs. I wouldn't use it if you use reliable delivery. It's good for newsletters because it has low cost compared to other services and when you don't care if some e-mails are not delivered.
>
> My recommendation is to setup a VPS (from a company that has clean network) with multiple IPs if you need to send a lot of messages and use postfix relay with randmap to balance the outgoing messages between the IPs.

Thank you for reply. Then I consider VPS instead of AWS EC2 and SES.

---
Yasuhiro KIMURA

Correct. I would recommend linode or digitalocean any time over AWS SES. AWS is a good option for heavy transactional mail alerts etc. 

Cheers/DP
Reply | Threaded
Open this post in threaded view
|

Re: Forwarding received mail through AWS SES

Antonio Leding
FWIW - I’ve been using AWS for outbound SMTP well over 5 years with no issues…maybe one-time have I bad an email rejected due to blacklisting…and this was resolved within 30 minutes…



On Jan 19, 2019, at 7:13 PM, Durga Prasad Malyala <[hidden email]> wrote:


On Sat, Jan 19, 2019, 23:26 Yasuhiro KIMURA <[hidden email] wrote:
From: Christos Chatzaras <[hidden email]>
Subject: Re: Forwarding received mail through AWS SES
Date: Sat, 19 Jan 2019 12:35:58 +0200

> AWS EC2 IPs may have low reputation to e-mail providers, so is not recommended to send e-mails using these IPs.
>
> Also AWS SES frequently have issues with RBLs. I wouldn't use it if you use reliable delivery. It's good for newsletters because it has low cost compared to other services and when you don't care if some e-mails are not delivered.
>
> My recommendation is to setup a VPS (from a company that has clean network) with multiple IPs if you need to send a lot of messages and use postfix relay with randmap to balance the outgoing messages between the IPs.

Thank you for reply. Then I consider VPS instead of AWS EC2 and SES.

---
Yasuhiro KIMURA

Correct. I would recommend linode or digitalocean any time over AWS SES. AWS is a good option for heavy transactional mail alerts etc. 

Cheers/DP

Reply | Threaded
Open this post in threaded view
|

Re: Forwarding received mail through AWS SES

Antonio Leding
Clarifying - I have both SES and EC2.  EC2 is my main postfix box but the SMTP side is a backup for SES which is my main outbound email…


On Jan 19, 2019, at 7:16 PM, Antonio Leding <[hidden email]> wrote:

FWIW - I’ve been using AWS for outbound SMTP well over 5 years with no issues…maybe one-time have I bad an email rejected due to blacklisting…and this was resolved within 30 minutes…



On Jan 19, 2019, at 7:13 PM, Durga Prasad Malyala <[hidden email]> wrote:


On Sat, Jan 19, 2019, 23:26 Yasuhiro KIMURA <[hidden email] wrote:
From: Christos Chatzaras <[hidden email]>
Subject: Re: Forwarding received mail through AWS SES
Date: Sat, 19 Jan 2019 12:35:58 +0200

> AWS EC2 IPs may have low reputation to e-mail providers, so is not recommended to send e-mails using these IPs.
>
> Also AWS SES frequently have issues with RBLs. I wouldn't use it if you use reliable delivery. It's good for newsletters because it has low cost compared to other services and when you don't care if some e-mails are not delivered.
>
> My recommendation is to setup a VPS (from a company that has clean network) with multiple IPs if you need to send a lot of messages and use postfix relay with randmap to balance the outgoing messages between the IPs.

Thank you for reply. Then I consider VPS instead of AWS EC2 and SES.

---
Yasuhiro KIMURA

Correct. I would recommend linode or digitalocean any time over AWS SES. AWS is a good option for heavy transactional mail alerts etc. 

Cheers/DP


Reply | Threaded
Open this post in threaded view
|

Re: Forwarding received mail through AWS SES

John Stoffel-2
In reply to this post by Durga Prasad Malyala
>>>>> "Durga" == Durga Prasad Malyala <[hidden email]> writes:


Durga> Correct. I would recommend linode or digitalocean any time over
Durga> AWS SES. AWS is a good option for heavy transactional mail
Durga> alerts etc. 

The only problem with Digital Ocean right now is that Charter/Spectrum
in the US has blocked all (most? At least the one I'm using...) blocks
assigned to DO for some insane reason.   I've sorta just given up
getting this fixed.  It's a spam mitigation (ha!) feature of
Chater/Spectrum.



Reply | Threaded
Open this post in threaded view
|

Re: Forwarding received mail through AWS SES

Yuval Levy
On 2019-01-20 14:40, John Stoffel wrote:
> The only problem with Digital Ocean right now is that Charter/Spectrum
> in the US has blocked all (most? At least the one I'm using...) blocks
> assigned to DO for some insane reason.

Why insane?  Having been a DO customer for more than five years, I am
not surprised.  *bad neighborhood*.  DO is not doing enough to prevent
spam and viruses emanating from its network.  This affects me both as a
sender and as a recipient of emails.

I have received too much spam emanating from DO's network, including
traditional email spam and text messages with links to droplets on the
DO network that were spreading malware.  When I tried to get DO's abuse
team to take action, they were slow, dismissive, useless.  Despite
evidence to a level that would withstand in a court of justice, they
closed the cases with no action taken simply because the offending
droplet has gone.  It does not seem to have occurred to them that the
spammer has probably gone to another IP address on their network.

My conclusion was to jump ship at the next opportunity.  I am currently
testing an alternative provider and will possibly say goodbye to DO in a
few weeks.


Reply | Threaded
Open this post in threaded view
|

Re: Forwarding received mail through AWS SES

John Stoffel-2
>>>>> "Yuval" == Yuval Levy <[hidden email]> writes:

Yuval> On 2019-01-20 14:40, John Stoffel wrote:
>> The only problem with Digital Ocean right now is that Charter/Spectrum
>> in the US has blocked all (most? At least the one I'm using...) blocks
>> assigned to DO for some insane reason.

Yuval> Why insane?  Having been a DO customer for more than five
Yuval> years, I am not surprised.  *bad neighborhood*.  DO is not
Yuval> doing enough to prevent spam and viruses emanating from its
Yuval> network.  This affects me both as a sender and as a recipient
Yuval> of emails.

I'm not sure you'll ever find *any* good neighborhood, since it's too
easy for anyone to spin up a system next to yours IP wise, and then
someone just takes the hammer and nukes the entire block.  Even if you
are a good netizen.

Yuval> I have received too much spam emanating from DO's network,
Yuval> including traditional email spam and text messages with links
Yuval> to droplets on the DO network that were spreading malware.
Yuval> When I tried to get DO's abuse team to take action, they were
Yuval> slow, dismissive, useless.  Despite evidence to a level that
Yuval> would withstand in a court of justice, they closed the cases
Yuval> with no action taken simply because the offending droplet has
Yuval> gone.  It does not seem to have occurred to them that the
Yuval> spammer has probably gone to another IP address on their
Yuval> network.

I don't bother too much with tracking it for my own personal domain, I
just postgrey, spamassassin, and then try to black hole those that
make it through.  It's not ideal, but I don't have a beter answer and
I don't think *anyone* does.  

Yuval> My conclusion was to jump ship at the next opportunity.  I am
Yuval> currently testing an alternative provider and will possibly say
Yuval> goodbye to DO in a few weeks.

Share the results please.

John

Reply | Threaded
Open this post in threaded view
|

Re: Forwarding received mail through AWS SES

Stephen Satchell
In reply to this post by Yasuhiro KIMURA
On 2019-01-20 14:40, John Stoffel wrote:
> The only problem with Digital Ocean right now is that Charter/Spectrum
> in the US has blocked all (most? At least the one I'm using...) blocks
> assigned to DO for some insane reason.

The insane reason is phishing spam, and DO ignoring abuse notices.

And this is not an appropriate subject for the Postfix mailing list.