virtual_alias_maps and X-Original-To

[adam hamer]

I have an entry in the virtual_alias_maps for a few users to be redirected to zendesk.com. zendesk requires a X-Original-To header set for some stuff to work, but it isn't added in my postfix setup.
I have a basic postfix setup with dovecot for virtual delivery, but from the logs it appears dovecot is not instantiated when using virtual_alias_maps since the alias means its forwarded - because its an smtp task?    It seems to make sense from this thread that       [http://www.dovecot.org/list/dovecot/2011-January/056783.html]       "postfix adds X-Original-To when delivering to a mailbox - which delivery via smtp/lmtp isn't.",    and this one       http://tech.groups.yahoo.com/group/postfix-users/message/231881       "Postfix will not add X-Original-To when forwarding mail."
It seems filters were suggested to be used, but it doesn't seem a common approach. I'd appreciate some guidance on this - I am a newbie trying hard not to get too lost...
Regards,adam
     

[Victor Duchovni]

On Thu, Feb 17, 2011 at 05:03:04PM +0000, Adam Hamer wrote:

> I have an entry in the virtual_alias_maps for a few users to be
> redirected to zendesk.com. zendesk requires a X-Original-To header set
> for some stuff to work, but it isn't added in my postfix setup.

> I have a basic postfix setup with dovecot for virtual delivery,
> but from the logs it appears dovecot is not instantiated when using
> virtual_alias_maps since the alias means its forwarded - because its
> an smtp task???

No, the aliasing is not the crux of the issue when the resulting recipient
is still delivered locally, but see below.

> It seems to make sense from this thread that:
>
> http://www.dovecot.org/list/dovecot/2011-January/056783.html
>
> "postfix adds X-Original-To when delivering to a mailbox
>
>   - which delivery via smtp/lmtp isn't.",

This is right, only the local(8), virtual(8) and pipe(8) (flag-dependent)
prepend X-Original-To. The smtp(8) and lmtp(8) delivery agents do not
prepend X-Original-To headers.

> and this one:
>
> http://tech.groups.yahoo.com/group/postfix-users/message/231881
>
> "Postfix will not add X-Original-To when forwarding mail."

Yes, really when not using a delivery agent that is typically used for
outbound or relay email. In the case of lmtp(8) this should perhaps be
revisited at some point.

> It seems filters were suggested to be used, but it doesn't seem a
> common approach. I'd appreciate some guidance on this - I am a newbie
> trying hard not to get too lost...

Are you in fact forwarding email off-site or are you using lmtp(8) to
deliver email into a Dovecot IMAP store?

--
        Viktor.

[adam hamer]

> On Thu, Feb 17, 2011 at 05:03:04PM +0000, Adam Hamer wrote:
>
> > I have an entry in the virtual_alias_maps for a few users to be
> > redirected to zendesk.com. zendesk requires a X-Original-To header set
> > for some stuff to work, but it isn't added in my postfix setup.
>
> > I have a basic postfix setup with dovecot for virtual delivery,
> > but from the logs it appears dovecot is not instantiated when using
> > virtual_alias_maps since the alias means its forwarded - because its
> > an smtp task???
>
> No, the aliasing is not the crux of the issue when the resulting recipient
> is still delivered locally, but see below.
>

The resulting recipient is not delivered locally - it goes to zendesk.com an email 'helpdesk' system. Essentially I have a domain Y.com which points to postfix. Postfix is configure to handle it as a virtual domain with dovecot, but some aliases are set to forward say [hidden email] to zendesk.com. I'm guessing those that get forwarded have nothing to do with local/mailbox delivery agent - and hence its smtp which handles it.

> > It seems to make sense from this thread that:
> >
> > http://www.dovecot.org/list/dovecot/2011-January/056783.html
> >
> > "postfix adds X-Original-To when delivering to a mailbox
> >
> >   - which delivery via smtp/lmtp isn't.",
>
> This is right, only the local(8), virtual(8) and pipe(8) (flag-dependent)
> prepend X-Original-To. The smtp(8) and lmtp(8) delivery agents do not
> prepend X-Original-To headers.
>
> > and this one:
> >
> > http://tech.groups.yahoo.com/group/postfix-users/message/231881
> >
> > "Postfix will not add X-Original-To when forwarding mail."
>
> Yes, really when not using a delivery agent that is typically used for
> outbound or relay email. In the case of lmtp(8) this should perhaps be
> revisited at some point.
> 

So my alias is outbound mail - I don't use lmtp (sorry - should have clarified that).
> > It seems filters were suggested to be used, but it doesn't seem a
> > common approach. I'd appreciate some guidance on this - I am a newbie
> > trying hard not to get too lost...
>
> Are you in fact forwarding email off-site or are you using lmtp(8) to
> deliver email into a Dovecot IMAP store?
>
Off-site. Any ideas for which way to go to make those aliases have the X-Original-To?  
> --
> Viktor.

Thanks,adam
     

[Victor Duchovni]

On Thu, Feb 17, 2011 at 09:43:05PM +0000, Adam Hamer wrote:

> > Are you in fact forwarding email off-site or are you using lmtp(8) to
> > deliver email into a Dovecot IMAP store?
>
> Off-site. Any ideas for which way to go to make those aliases have
> the X-Original-To?

Aliasing via virtual(5) is brutally efficient. No content modification,
just message routing, so the header is not added.

To add an X-Original-Recipient header, messages to multiple recipients
have to be split into one copy per-recipient since the header you want
is recipient-specific.

Since the queue file contains a single copy of the message for all
recipients, such rewriting *cannot* happen on input, it can only happen on
output via a delivery agent that is processing a single recipient. This
means you need a custom transport for this destination that processes
mail one recipient at a time and prepends the header.

This can be done via an external program via pipe(8) or by modifying the
source code of the smtp(8) delivery agent so that the header is added when
a suitable boolean flag is set and the message has exactly one recipient.

The pipe(8) approach would then re-submit the mail for outbound delivery
to a different Postfix instance to avoid loops.

--
        Viktor.

[adam hamer]

> Aliasing via virtual(5) is brutally efficient. No content modification,
> just message routing, so the header is not added.
>
> To add an X-Original-Recipient header, messages to multiple recipients
> have to be split into one copy per-recipient since the header you want
> is recipient-specific.
>
> Since the queue file contains a single copy of the message for all
> recipients, such rewriting *cannot* happen on input, it can only happen on
> output via a delivery agent that is processing a single recipient. This
> means you need a custom transport for this destination that processes
> mail one recipient at a time and prepends the header.
>
> This can be done via an external program via pipe(8) or by modifying the
> source code of the smtp(8) delivery agent so that the header is added when
> a suitable boolean flag is set and the message has exactly one recipient.
>
> The pipe(8) approach would then re-submit the mail for outbound delivery
> to a different Postfix instance to avoid loops.
>
> --
> Viktor.

  Thanks again Viktor.
It sounds a bit more in depth than I was hoping for. Given the _only_ function for this alias is to forward to one address, would it be possible to do a simpler approach? 
Regards, adam

     

[Wietse Venema]

Adam Hamer:
[ Charset ISO-8859-1 unsupported, converting... ]

>
>
>
> > Aliasing via virtual(5) is brutally efficient. No content modification,
> > just message routing, so the header is not added.
> >
> > To add an X-Original-Recipient header, messages to multiple recipients
> > have to be split into one copy per-recipient since the header you want
> > is recipient-specific.
> >
> > Since the queue file contains a single copy of the message for all
> > recipients, such rewriting *cannot* happen on input, it can only happen on
> > output via a delivery agent that is processing a single recipient. This
> > means you need a custom transport for this destination that processes
> > mail one recipient at a time and prepends the header.
> >
> > This can be done via an external program via pipe(8) or by modifying the
> > source code of the smtp(8) delivery agent so that the header is added when
> > a suitable boolean flag is set and the message has exactly one recipient.
> >
> > The pipe(8) approach would then re-submit the mail for outbound delivery
> > to a different Postfix instance to avoid loops.
> >
> > --
> > Viktor.
>
> ??Thanks again Viktor.
> It sounds a bit more in depth than I was hoping for. Given the _only_ func
>-tion for this alias is to forward to one address, would it be possible to do
>- a simpler approach??

If it is only for one special case,

/etc/postfix/main.cf
    smtpd_recipient_restrictions =
        permit_mynetworks
        ...
        reject_unauth_destination
        ...
        check_recipient_access hash:/etc/postfix/recipient_access

/etc/postfix/recipient_access:
    [hidden email] PREPEND: X-Original-To: [hidden email]

If it is for EXIM emulation:

/etc/postfix/main.cf
    smtpd_recipient_restrictions =
        permit_mynetworks
        ...
        reject_unauth_destination
        ...
        check_recipient_access pcre:/etc/postfix/recipient_access.pcre

/etc/postfix/recipient_access.pcre:
    /(.+)/ PREPEND: X-Envelope-To: $1

Either way, if mail has multiple recipients, then they will see
the prepended message header.

        Wietse

[Wietse Venema]

Wietse Venema:

> > ??Thanks again Viktor.
> > It sounds a bit more in depth than I was hoping for. Given the _only_ func
> >-tion for this alias is to forward to one address, would it be possible to do
> >- a simpler approach??
>
> If it is only for one special case,
>
> /etc/postfix/main.cf
>     smtpd_recipient_restrictions =
> permit_mynetworks
> ...
> reject_unauth_destination
> ...
> check_recipient_access hash:/etc/postfix/recipient_access
>
> /etc/postfix/recipient_access:
>     [hidden email] PREPEND: X-Original-To: [hidden email]
>
> If it is for EXIM emulation:
>
> /etc/postfix/main.cf
>     smtpd_recipient_restrictions =
>         permit_mynetworks
>         ...
>         reject_unauth_destination
>         ...
> check_recipient_access pcre:/etc/postfix/recipient_access.pcre
>
> /etc/postfix/recipient_access.pcre:
>     /(.+)/ PREPEND: X-Envelope-To: $1
>
> Either way, if mail has multiple recipients, then they will see
> the prepended message header.

Perhaps you want to place the check_recipient_access before the
permit_mynetworks, so that the header is also prepended applies
when local client send mail.

        Wietse

[adam hamer]

> Wietse Venema:>> If it is only for one special case,
>
> /etc/postfix/main.cf
> smtpd_recipient_restrictions =
> permit_mynetworks
> ...
> reject_unauth_destination
> ...
> check_recipient_access hash:/etc/postfix/recipient_access
>
> /etc/postfix/recipient_access:
> [hidden email] PREPEND: X-Original-To: [hidden email]
>> Wietse

Just wanted to say thank you for the info. It turned out I needed "PREPEND" not "PREPEND:" (so, without the colon) - perhaps a different version or something, but it works!
Thanks,adam
     

[Charles Marcus]

On 2011-02-17 1:16 PM, Victor Duchovni wrote:
> On Thu, Feb 17, 2011 at 05:03:04PM +0000, Adam Hamer wrote:
>> "Postfix will not add X-Original-To when forwarding mail."

> Yes, really when not using a delivery agent that is typically used
> for outbound or relay email. In the case of lmtp(8) this should
> perhaps be revisited at some point.

Just to make sure I understand this correctly...

Currently, using virtual for deliver, I'm getting these X-Original-To
headers, which I have gotten very used to and rely on - but if/when we
switch to dovecot+LMTP, we will lose them?

If so, that is a show-stopper for me... also, what are the chances of
this being 'revisited' at any point in the foreseeable future?

Last - am I correct that we will *not* lose them if we just use the
dovecot LDA as opposed to LMTP?

Thanks,

--

Best regards,

Charles

[Wietse Venema]

Charles Marcus:

> On 2011-02-17 1:16 PM, Victor Duchovni wrote:
> > On Thu, Feb 17, 2011 at 05:03:04PM +0000, Adam Hamer wrote:
> >> "Postfix will not add X-Original-To when forwarding mail."
>
> > Yes, really when not using a delivery agent that is typically used
> > for outbound or relay email. In the case of lmtp(8) this should
> > perhaps be revisited at some point.
>
> Just to make sure I understand this correctly...
>
> Currently, using virtual for deliver, I'm getting these X-Original-To
> headers, which I have gotten very used to and rely on - but if/when we
> switch to dovecot+LMTP, we will lose them?

You can fake it in the SMTP server with

/etc/postfix/main.cf:
    smtpd_recipient_restrictions =
        check_recipient_access pcre:/etc/postfix/recipient_access.pcre

/etc/postfix/recipient_access.pcre
    /(.+)/ prepend X-Original-To: $1

At a minor loss of privacy (plus that it would prepend
multiple headers in the case of multi-recipient mail).

> If so, that is a show-stopper for me... also, what are the chances of
> this being 'revisited' at any point in the foreseeable future?
>
> Last - am I correct that we will *not* lose them if we just use the
> dovecot LDA as opposed to LMTP?

The chances depend on available time. Implementing one thing means
delaying another.

        Wietse

[Charles Marcus]

On 2011-03-01 3:35 PM, Wietse Venema wrote:
> Charles Marcus:
>> Currently, using virtual for deliver, I'm getting these X-Original-To
>> headers, which I have gotten very used to and rely on - but if/when we
>> switch to dovecot+LMTP, we will lose them?

> You can fake it in the SMTP server with
<snip>
> At a minor loss of privacy (plus that it would prepend
> multiple headers in the case of multi-recipient mail).

Thanks very much for the workaround... I'll experiment and see if the
privacy implications are anything we should worry about (most people
don't even know what headers are, much less how to look at them, so
probably not a big deal)...

>> If so, that is a show-stopper for me... also, what are the chances of
>> this being 'revisited' at any point in the foreseeable future?

> The chances depend on available time. Implementing one thing means
> delaying another.

Understood... hopefully it is at least on a To-Do somewhere so it won't
get lost in the shuffle.

Thanks again Wietse!

--

Best regards,

Charles