Address rewriting not working

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Address rewriting not working

Celejar
Hi,

I'm configuring Postfix to relay mail via a smarthost, and I need to
rewrite the sender address in order for the smarthost to accept the
mail (and not reject it as 'relaying'). I'm using generic mapping to do
this, and it works correctly on two of my systems (Debian Sid,
running Postfix 3.3.2), but not on a third (Debian Stretch, running
3.1.8). I've tried all sorts of adjustments and debugging, and I'm at
my wits' end. Below is the configuration and logging from the broken
system:

From /var/log/mail.log :

Jan  2 14:05:55 alice postfix/pickup[17200]: D2324AC400BA: uid=0 from=<root>
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: name_mask: all
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: inet_addr_local: configured 2 IPv4 addresses
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: inet_addr_local: configured 5 IPv6 addresses
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: process generation: 20 (20)
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: connection established fd 128
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: master_notify: status 0
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: request
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute name: request
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute value: rewrite
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: rule
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute name: rule
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute value: local
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: address
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute name: address
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute value: root
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: (list terminator)
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute name: (end)
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: `local' `root' -> `[hidden email]'
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: send attr flags = 0
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: send attr address = [hidden email]
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: master_notify: status 1
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: master_notify: status 0
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: request
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute name: request
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute value: rewrite
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: rule
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute name: rule
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute value: local
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: address
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute name: address
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute value: [hidden email]
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: (list terminator)
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute name: (end)
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: `local' `[hidden email]' -> `[hidden email]'
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: send attr flags = 0
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: send attr address = [hidden email]
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: master_notify: status 1
Jan  2 14:05:55 alice postfix/cleanup[19935]: D2324AC400BA: message-id=<[hidden email]>
Jan  2 14:05:55 alice postfix/qmgr[17048]: D2324AC400BA: from=<[hidden email]>, size=258, nrcpt=1 (queue active)
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: connection established fd 129
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: master_notify: status 0
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: request
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute name: request
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute value: resolve
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: sender
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute name: sender
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute value: [hidden email]
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: address
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute name: address
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute value: [hidden email]
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: (list terminator)
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: input attribute name: (end)
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: match_string: mydestination: gmail.com ~? alice.villette
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: match_string: mydestination: gmail.com ~? alice.villette
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: match_string: mydestination: gmail.com ~? localhost.villette
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: match_string: mydestination: gmail.com ~? localhost
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: match_list_match: gmail.com: no match
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: `[hidden email]' -> `[hidden email]' -> (`smtp' `[smtp.zoho.com]:465' `[hidden email]' `4096')
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: send attr flags = 0
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: send attr transport = smtp
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: send attr nexthop = [smtp.zoho.com]:465
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: send attr recipient = [hidden email]
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: send attr flags = 4096
Jan  2 14:05:55 alice postfix/trivial-rewrite[19936]: master_notify: status 1
Jan  2 14:05:56 alice postfix/trivial-rewrite[19936]: connection established fd 130
Jan  2 14:05:56 alice postfix/trivial-rewrite[19936]: master_notify: status 0
Jan  2 14:05:56 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: request
Jan  2 14:05:56 alice postfix/trivial-rewrite[19936]: input attribute name: request
Jan  2 14:05:56 alice postfix/trivial-rewrite[19936]: input attribute value: rewrite
Jan  2 14:05:56 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: rule
Jan  2 14:05:56 alice postfix/trivial-rewrite[19936]: input attribute name: rule
Jan  2 14:05:56 alice postfix/trivial-rewrite[19936]: input attribute value: local
Jan  2 14:05:56 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: address
Jan  2 14:05:56 alice postfix/trivial-rewrite[19936]: input attribute name: address
Jan  2 14:05:56 alice postfix/trivial-rewrite[19936]: input attribute value: [hidden email]
Jan  2 14:05:56 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: (list terminator)
Jan  2 14:05:56 alice postfix/trivial-rewrite[19936]: input attribute name: (end)
Jan  2 14:05:56 alice postfix/trivial-rewrite[19936]: `local' `[hidden email]' -> `[hidden email]'
Jan  2 14:05:56 alice postfix/trivial-rewrite[19936]: send attr flags = 0
Jan  2 14:05:56 alice postfix/trivial-rewrite[19936]: send attr address = [hidden email]
Jan  2 14:05:56 alice postfix/trivial-rewrite[19936]: master_notify: status 1
Jan  2 14:05:57 alice postfix/smtp[19937]: D2324AC400BA: to=<[hidden email]>, relay=smtp.zoho.com[204.141.43.189]:465, delay=1.4, delays=0.11/0.03/0.88/0.4, dsn=5.0.0, status=bounced (host smtp.zoho.com[204.141.43.189] said: 553 Relaying disallowed as @ (in reply to end of DATA command))
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: master_notify: status 0
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: request
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute name: request
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute value: rewrite
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: rule
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute name: rule
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute value: local
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: address
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute name: address
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute value: ""
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: (list terminator)
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute name: (end)
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: `local' `""' -> `""'
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: send attr flags = 0
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: send attr address = ""
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: master_notify: status 1
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: master_notify: status 0
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: request
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute name: request
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute value: rewrite
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: rule
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute name: rule
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute value: local
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: address
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute name: address
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute value: [hidden email]
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: (list terminator)
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute name: (end)
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: `local' `[hidden email]' -> `[hidden email]'
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: send attr flags = 0
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: send attr address = [hidden email]
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: master_notify: status 1
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: master_notify: status 0
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: request
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute name: request
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute value: rewrite
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: rule
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute name: rule
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute value: local
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: address
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute name: address
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute value: MAILER-DAEMON
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: (list terminator)
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute name: (end)
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: `local' `MAILER-DAEMON' -> `[hidden email]'
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: send attr flags = 0
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: send attr address = [hidden email]
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: master_notify: status 1
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: master_notify: status 0
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: request
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute name: request
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute value: rewrite
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: rule
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute name: rule
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute value: local
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: address
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute name: address
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute value: [hidden email]
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: (list terminator)
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute name: (end)
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: `local' `[hidden email]' -> `[hidden email]'
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: send attr flags = 0
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: send attr address = [hidden email]
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: master_notify: status 1
Jan  2 14:05:57 alice postfix/cleanup[19935]: 40739AC400BB: message-id=<[hidden email]>
Jan  2 14:05:57 alice postfix/bounce[19939]: D2324AC400BA: sender non-delivery notification: 40739AC400BB
Jan  2 14:05:57 alice postfix/qmgr[17048]: 40739AC400BB: from=<>, size=2235, nrcpt=1 (queue active)
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: master_notify: status 0
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: request
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute name: request
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute value: resolve
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: sender
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute name: sender
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute value: (end)
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: address
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute name: address
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute value: [hidden email]
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: rewrite socket: wanted attribute: (list terminator)
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: input attribute name: (end)
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: match_string: mydestination: alice.villette ~? alice.villette
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: `' -> `[hidden email]' -> (`local' `alice.villette' `[hidden email]' `256')
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: send attr flags = 0
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: send attr transport = local
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: send attr nexthop = alice.villette
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: send attr recipient = [hidden email]
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: send attr flags = 256
Jan  2 14:05:57 alice postfix/trivial-rewrite[19936]: master_notify: status 1
Jan  2 14:05:57 alice postfix/qmgr[17048]: D2324AC400BA: removed
Jan  2 14:05:57 alice postfix/local[19940]: 40739AC400BB: to=<[hidden email]>, relay=local, delay=0.09, delays=0.04/0.01/0/0.04, dsn=2.0.0, status=sent (delivered to mailbox)
Jan  2 14:05:57 alice postfix/qmgr[17048]: 40739AC400BB: removed
Jan  2 14:06:01 alice postfix/trivial-rewrite[19936]: connection closed fd 130
Jan  2 14:06:02 alice postfix/trivial-rewrite[19936]: connection closed fd 128
Jan  2 14:06:02 alice postfix/trivial-rewrite[19936]: connection closed fd 129
Jan  2 14:07:42 alice postfix/trivial-rewrite[19936]: idle timeout -- exiting

# postconf -n
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
append_dot_mydomain = no
biff = no
compatibility_level = 2
inet_interfaces = all
inet_protocols = all
mailbox_size_limit = 0
mydestination = $myhostname, alice.villette, localhost.villette, localhost
myhostname = alice.villette
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
myorigin = /etc/mailname
readme_directory = no
recipient_delimiter = +
relayhost = [smtp.zoho.com]:465
smtp_generic_maps = hash:/etc/postfix/generic
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/zoho_passwd
smtp_sasl_security_options =
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtp_tls_security_level = verify
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_tls_wrappermode = yes
smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
smtpd_tls_cert_file = /etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file = /etc/ssl/private/ssl-cert-snakeoil.key
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtpd_use_tls = yes

# postmap -s generic
@alice.villette [hidden email]

# postconf -Mf
smtp       inet  n       -       y       -       -       smtpd
pickup     unix  n       -       y       60      1       pickup
cleanup    unix  n       -       y       -       0       cleanup
qmgr       unix  n       -       n       300     1       qmgr
tlsmgr     unix  -       -       y       1000?   1       tlsmgr
rewrite    unix  -       -       y       -       -       trivial-rewrite -v
bounce     unix  -       -       y       -       0       bounce
defer      unix  -       -       y       -       0       bounce
trace      unix  -       -       y       -       0       bounce
verify     unix  -       -       y       -       1       verify
flush      unix  n       -       y       1000?   0       flush
proxymap   unix  -       -       n       -       -       proxymap
proxywrite unix  -       -       n       -       1       proxymap
smtp       unix  -       -       y       -       -       smtp
relay      unix  -       -       y       -       -       smtp
showq      unix  n       -       y       -       -       showq
error      unix  -       -       y       -       -       error
retry      unix  -       -       y       -       -       error
discard    unix  -       -       y       -       -       discard
local      unix  -       n       n       -       -       local
virtual    unix  -       n       n       -       -       virtual
lmtp       unix  -       -       y       -       -       lmtp
anvil      unix  -       -       y       -       1       anvil
scache     unix  -       -       y       -       1       scache
maildrop   unix  -       n       n       -       -       pipe flags=DRhu
    user=vmail argv=/usr/bin/maildrop -d ${recipient}
uucp       unix  -       n       n       -       -       pipe flags=Fqhu
    user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
ifmail     unix  -       n       n       -       -       pipe flags=F user=ftn
    argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
bsmtp      unix  -       n       n       -       -       pipe flags=Fq.
    user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
scalemail-backend unix - n       n       -       2       pipe flags=R
    user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop}
    ${user} ${extension}
mailman    unix  -       n       n       -       -       pipe flags=FR
    user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py ${nexthop}
    ${user}

Celejar
Reply | Threaded
Open this post in threaded view
|

Re: Address rewriting not working

Viktor Dukhovni
> On Jan 2, 2019, at 7:12 PM, Celejar <[hidden email]> wrote:
>
> I'm configuring Postfix to relay mail via a smarthost, and I need to
> rewrite the sender address in order for the smarthost to accept the
> mail (and not reject it as 'relaying'). I'm using generic mapping to do
> this, and it works correctly on two of my systems (Debian Sid,
> running Postfix 3.3.2), but not on a third (Debian Stretch, running
> 3.1.8). I've tried all sorts of adjustments and debugging, and I'm at
> my wits' end. Below is the configuration and logging from the broken
> system:

smtp_generic_maps (a.k.a. generic_maps) is (not surprisingly), done
in the smtp(8) delivery agent in real-time as the message is being
sent, and not in trivial-rewrite, which only resolves addresses
to standard form and resolves to a transport.  Hence the "trivial"
in the name.

Pre-queue rewriting (masquerading, canonical_maps and virtual_alias_maps
for just the recipient) happens in cleanup(8).  Post-queue rewriting
happens in delivery agents, with just transport resolution in trivial
rewrite called from the queue-manager in between.

Because the queue manager logs the envelope sender prior to delivery,
it always logs the original value, and recipient logging in delivery
agents is also the form before generic rewriting.  The output of
generic rewriting is not logged on the sending system (except perhaps
in verbose logging that should not normally be enabled).

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: Address rewriting not working

Celejar
On Wed, 2 Jan 2019 19:58:18 -0500
Viktor Dukhovni <[hidden email]> wrote:

> > On Jan 2, 2019, at 7:12 PM, Celejar <[hidden email]> wrote:
> >
> > I'm configuring Postfix to relay mail via a smarthost, and I need to
> > rewrite the sender address in order for the smarthost to accept the
> > mail (and not reject it as 'relaying'). I'm using generic mapping to do
> > this, and it works correctly on two of my systems (Debian Sid,
> > running Postfix 3.3.2), but not on a third (Debian Stretch, running
> > 3.1.8). I've tried all sorts of adjustments and debugging, and I'm at
> > my wits' end. Below is the configuration and logging from the broken
> > system:
>
> smtp_generic_maps (a.k.a. generic_maps) is (not surprisingly), done
> in the smtp(8) delivery agent in real-time as the message is being
> sent, and not in trivial-rewrite, which only resolves addresses
> to standard form and resolves to a transport.  Hence the "trivial"
> in the name.
>
> Pre-queue rewriting (masquerading, canonical_maps and virtual_alias_maps
> for just the recipient) happens in cleanup(8).  Post-queue rewriting
> happens in delivery agents, with just transport resolution in trivial
> rewrite called from the queue-manager in between.
>
> Because the queue manager logs the envelope sender prior to delivery,
> it always logs the original value, and recipient logging in delivery
> agents is also the form before generic rewriting.  The output of
> generic rewriting is not logged on the sending system (except perhaps
> in verbose logging that should not normally be enabled).

Ah, okay - thanks for clearing this up. I was following the
instructions in DEBUG_README, which say:

To diagnose problems with address rewriting specify a "-v" option for
the cleanup(8) and/or trivial-rewrite(8) daemon

and the ADDRESS_REWRITING_README classifies "Generic mapping for
outgoing SMTP mail" as a form of "Address rewriting when mail is
delivered".

I suggest that the instructions in DEBUG_README be clarified to
distinguish between the different forms of address rewriting.

As to the underlying problem, I may have figured it out: I switched
to using canonical mapping, since that gets properly logged, and I saw
that the rewrite is indeed happening correctly on all systems - but one
still has the mail rejected, while the others get it accepted. I
suspected that the problem may be in the way the email 'From:' header is
being rewritten, and I used smtp_header_checks to investigate. Sure
enough, the rewrite is happening differently on the two systems:

Here's the bad system:

Jan  3 15:09:10 alice postfix/qmgr[17481]: B0A33AC40093: from=<[hidden email]>, size=257, nrcpt=1 (queue active)
Jan  3 15:09:12 alice postfix/smtp[17489]: B0A33AC40093: warning: header From: [hidden email] (root)
Jan  3 15:09:12 alice postfix/smtp[17489]: B0A33AC40093: to=<[hidden email]>, relay=smtp.zoho.com[204.141.43.189]:465, delay=1.5, delays=0.1/0.1/0.88/0.4, dsn=5.0.0, status=bounced (host smtp.zoho.com[204.141.43.189] said: 553 Relaying disallowed as @ (in reply to end of DATA command))

And here's one of the good ones:

Jan  3 15:14:58 lila postfix/qmgr[14708]: 9AD005E18AD: from=<[hidden email]>, size=249, nrcpt=1 (queue active)
Jan  3 15:15:00 lila postfix/smtp[14715]: 9AD005E18AD: warning: header From: root <[hidden email]>
Jan  3 15:15:01 lila postfix/smtp[14715]: 9AD005E18AD: to=<[hidden email]>, relay=smtp.zoho.com[204.141.43.189]:465, delay=2.8, delays=0.27/0.06/1.4/1.1, dsn=2.0.0, status=sent (250 Message received)

IIUC, Zoho is being picky and rejecting the first form of the rewritten
'From:' header, with ' (root)' at the end. Does this seem right? I'm no
email expert, but that certainly seems a somewhat uncommon form. How
can I get Postfix do do something more standard?

Celejar
Reply | Threaded
Open this post in threaded view
|

[Partially solved] Re: Address rewriting not working

Celejar
In reply to this post by Viktor Dukhovni
On Wed, 2 Jan 2019 19:58:18 -0500
Viktor Dukhovni <[hidden email]> wrote:

> > On Jan 2, 2019, at 7:12 PM, Celejar <[hidden email]> wrote:
> >
> > I'm configuring Postfix to relay mail via a smarthost, and I need to
> > rewrite the sender address in order for the smarthost to accept the
> > mail (and not reject it as 'relaying'). I'm using generic mapping to do
> > this, and it works correctly on two of my systems (Debian Sid,
> > running Postfix 3.3.2), but not on a third (Debian Stretch, running
> > 3.1.8). I've tried all sorts of adjustments and debugging, and I'm at
> > my wits' end. Below is the configuration and logging from the broken
> > system:
>

> Because the queue manager logs the envelope sender prior to delivery,
> it always logs the original value, and recipient logging in delivery
> agents is also the form before generic rewriting.  The output of
> generic rewriting is not logged on the sending system (except perhaps
> in verbose logging that should not normally be enabled).

Okay, by testing with swaks I've confirmed the suspicion that I broached
in my previous mail: on the problematic system, the rewrite of the email
header 'From: root' is to 'From: <[hidden email]> (root)', which
causes the mail to be rejected by Zoho's server with '553 Relaying
disallowed as @'. On the working systems, the rewrite is to a more
normal 'From: root <[hidden email]>', which Zoho accepts.

So: is this a bug? Is there some way I can get Postfix 3.1.8 to do the
rewriting the normal way, like 3.3.2 does, or do I just need to upgrade
Postfix?

Celejar
Reply | Threaded
Open this post in threaded view
|

Re: [Partially solved] Re: Address rewriting not working

Wietse Venema
Celejar:

> Okay, by testing with swaks I've confirmed the suspicion that I broached
> in my previous mail: on the problematic system, the rewrite of the email
> header 'From: root' is to 'From: <[hidden email]> (root)', which
> causes the mail to be rejected by Zoho's server with '553 Relaying
> disallowed as @'. On the working systems, the rewrite is to a more
> normal 'From: root <[hidden email]>', which Zoho accepts.
>
> So: is this a bug? Is there some way I can get Postfix 3.1.8 to do the
> rewriting the normal way, like 3.3.2 does, or do I just need to upgrade
> Postfix?

See: http://www.postfix.org/postconf.5.htnl#header_from_format

        Wietse

header_from_format (default: standard)
       The  format of the Postfix-generated From: header. This setting affects
       the appearance of 'full name' information when a local program such  as
       /bin/mail  submits  a  message without From: header through the Postfix
       sendmail(1) command.

       Specify one of the following:

       standard (default)
              Produce a header formatted as "From: name <address>".   This  is
              the default as of Postfix 3.3.

       obsolete
              Produce  a  header  formatted as "From: address (name)". This is
              the behavior prior to Postfix 3.3.

       Notes:

       o      Postfix generates the format "From: address" when name  informa-
              tion  is  unavailable  or  the envelope sender address is empty.
              This is the same behavior as prior to Postfix 3.3.

       o      In the standard form, the name will be  quoted  if  it  contains
              specials  as defined in RFC 5322, or the "!%" address operators.

       o      The Postfix sendmail(1) command gets name information  from  the
              -F  command-line  option, from the NAME environment variable, or
              from the UNIX password file.

       This feature is available in Postfix 3.3 and later.

Reply | Threaded
Open this post in threaded view
|

Re: [Partially solved] Re: Address rewriting not working

Celejar
On Fri, 4 Jan 2019 13:19:10 -0500 (EST)
Wietse Venema <[hidden email]> wrote:

> Celejar:
> > Okay, by testing with swaks I've confirmed the suspicion that I broached
> > in my previous mail: on the problematic system, the rewrite of the email
> > header 'From: root' is to 'From: <[hidden email]> (root)', which
> > causes the mail to be rejected by Zoho's server with '553 Relaying
> > disallowed as @'. On the working systems, the rewrite is to a more
> > normal 'From: root <[hidden email]>', which Zoho accepts.
> >
> > So: is this a bug? Is there some way I can get Postfix 3.1.8 to do the
> > rewriting the normal way, like 3.3.2 does, or do I just need to upgrade
> > Postfix?
>
> See: http://www.postfix.org/postconf.5.htnl#header_from_format

...

> header_from_format (default: standard)

...

>        Specify one of the following:
>
>        standard (default)
>               Produce a header formatted as "From: name <address>".   This  is
>               the default as of Postfix 3.3.
>
>        obsolete
>               Produce  a  header  formatted as "From: address (name)". This is
>               the behavior prior to Postfix 3.3.

...

>        This feature is available in Postfix 3.3 and later.

And I'm using 3.1.8, where the rewriting isn't acceptable to my mail
provider, and this feature isn't available ;) So I guess I'm stuck,
unless I can upgrade Postfix?

Thanks for the pointer,
Celejar
Reply | Threaded
Open this post in threaded view
|

Re: [Partially solved] Re: Address rewriting not working

Viktor Dukhovni
> On Jan 4, 2019, at 2:56 PM, Celejar <[hidden email]> wrote:
>
> And I'm using 3.1.8, where the rewriting isn't acceptable to my mail
> provider, and this feature isn't available ;) So I guess I'm stuck,
> unless I can upgrade Postfix?

Your other option, if possible, is to inject email into Postfix
with the "From:" header already constructed the way you want.
Postfix only adds "From:" headers when missing.

You could also use a PCRE REPLACE rule in smtp_header_checks:

        # Replace some legacy "address (display name)" forms with a
        # more modern "display name <address>" form.
        /^From:\s*([^\s<>]+)\s+\(([^"]*)\)\s*$/ REPLACE From: "$2" <$1>

The server that's accepting "From: display name <address>"
and rejecting "From: address (display name)" is not blameless
It should be able to process either form.

--
--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: [Partially solved] Re: Address rewriting not working

Wietse Venema
In reply to this post by Celejar
Celejar:

> On Fri, 4 Jan 2019 13:19:10 -0500 (EST)
> Wietse Venema <[hidden email]> wrote:
>
> > Celejar:
> > > Okay, by testing with swaks I've confirmed the suspicion that I broached
> > > in my previous mail: on the problematic system, the rewrite of the email
> > > header 'From: root' is to 'From: <[hidden email]> (root)', which
> > > causes the mail to be rejected by Zoho's server with '553 Relaying
> > > disallowed as @'. On the working systems, the rewrite is to a more
> > > normal 'From: root <[hidden email]>', which Zoho accepts.
> > >
> > > So: is this a bug? Is there some way I can get Postfix 3.1.8 to do the
> > > rewriting the normal way, like 3.3.2 does, or do I just need to upgrade
> > > Postfix?
> >
> > See: http://www.postfix.org/postconf.5.htnl#header_from_format
> >        This feature is available in Postfix 3.3 and later.
>
> And I'm using 3.1.8, where the rewriting isn't acceptable to my mail
> provider, and this feature isn't available ;) So I guess I'm stuck,
> unless I can upgrade Postfix?

Can you provide the From: header *before* mail enters Postfix?

Otherwise you will have to update to Postfix 3.3.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: [Partially solved] Re: Address rewriting not working

Celejar
On Fri, 4 Jan 2019 15:22:08 -0500 (EST)
Wietse Venema <[hidden email]> wrote:

> Celejar:
> > On Fri, 4 Jan 2019 13:19:10 -0500 (EST)
> > Wietse Venema <[hidden email]> wrote:
> >
> > > Celejar:
> > > > Okay, by testing with swaks I've confirmed the suspicion that I broached
> > > > in my previous mail: on the problematic system, the rewrite of the email
> > > > header 'From: root' is to 'From: <[hidden email]> (root)', which
> > > > causes the mail to be rejected by Zoho's server with '553 Relaying
> > > > disallowed as @'. On the working systems, the rewrite is to a more
> > > > normal 'From: root <[hidden email]>', which Zoho accepts.
> > > >
> > > > So: is this a bug? Is there some way I can get Postfix 3.1.8 to do the
> > > > rewriting the normal way, like 3.3.2 does, or do I just need to upgrade
> > > > Postfix?
> > >
> > > See: http://www.postfix.org/postconf.5.htnl#header_from_format
> > >        This feature is available in Postfix 3.3 and later.
> >
> > And I'm using 3.1.8, where the rewriting isn't acceptable to my mail
> > provider, and this feature isn't available ;) So I guess I'm stuck,
> > unless I can upgrade Postfix?
>
> Can you provide the From: header *before* mail enters Postfix?
>
> Otherwise you will have to update to Postfix 3.3.

I suppose I might be able to look at individual mail sending
applications to see what they can do (cron, etc.), but it will probably
be simpler and more robust to just try to use a newer Postfix. I can't
easily upgrade the version on the current system, since it's running
Debian Stable, and the version from Unstable has unavailable
dependencies, so I'll probably just run a more recent Postfix on a
virtual machine.

Thanks for the help,

Celejar
Reply | Threaded
Open this post in threaded view
|

Re: [Partially solved] Re: Address rewriting not working

Celejar
In reply to this post by Viktor Dukhovni
On Fri, 4 Jan 2019 15:20:08 -0500
Viktor Dukhovni <[hidden email]> wrote:

> > On Jan 4, 2019, at 2:56 PM, Celejar <[hidden email]> wrote:
> >
> > And I'm using 3.1.8, where the rewriting isn't acceptable to my mail
> > provider, and this feature isn't available ;) So I guess I'm stuck,
> > unless I can upgrade Postfix?
>
> Your other option, if possible, is to inject email into Postfix
> with the "From:" header already constructed the way you want.
> Postfix only adds "From:" headers when missing.
>
> You could also use a PCRE REPLACE rule in smtp_header_checks:
>
> # Replace some legacy "address (display name)" forms with a
> # more modern "display name <address>" form.
> /^From:\s*([^\s<>]+)\s+\(([^"]*)\)\s*$/ REPLACE From: "$2" <$1>

Thanks much - this seems to be working.

> The server that's accepting "From: display name <address>"
> and rejecting "From: address (display name)" is not blameless
> It should be able to process either form.

Of course. It's just that I have higher expectations of quality open
source software than I do of a commercial mail provider ;)

Thanks again for all the help,

Celejar