Virtual alias address class and no_address_mappings

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Virtual alias address class and no_address_mappings

Peter Ajamian
I'm not entirely certain if this is intentional or not, but I ran across
this one with someone in IRC just now.

If someone uses virtual_address_domains and has
"receive_override_options = no_address_mappings", then postfix will kick
back an error of "User unknown in virtual alias table" for any recipient
in that domain.  This is probably because postfix doesn't load or access
virtual_alias_maps if no_address_mappings is set.  This can be
problematic in the following scenario:

User has a content filter configured which re-injects to postfix on
another port which does not have no_address_mappings set.  The intent
here is for address mappings to be ignored and the content filter run
initially then upon re-injection for the address mappings to be
processed and the message routed as appropriate.  Because
no_address_mappings is set and the recipient address is in the virtual
alias address class, however, the message instead gets rejected
initially with the above mentioned error.

A better approach might be to read the virtual_alias_maps in order to
allow the message to pass the recipient check but to still not use it
for address mappings if no_address_mappings is set.

Is this behavior intentional or just a side effect of how
no_address_mappings is implemented?


Peter
Reply | Threaded
Open this post in threaded view
|

Re: Virtual alias address class and no_address_mappings

Viktor Dukhovni
On Mon, Dec 30, 2019 at 04:37:32PM +1300, Peter wrote:

> If someone uses virtual_address_domains and has
> "receive_override_options = no_address_mappings", then postfix will
> kick back an error of "User unknown in virtual alias table" for any
> recipient in that domain.

Addresses in virtual_alias_domains resolve to the error transport,
unless rewritten to a "real domain", typically via virtual_alias_maps.

But, content_filter bypasses transport resolution, since all recipients
go to the filter transport, so the below is not what typically happens.

> User has a content filter configured which re-injects to postfix on
> another port which does not have no_address_mappings set.  The intent
> here is for address mappings to be ignored and the content filter run
> initially then upon re-injection for the address mappings to be
> processed and the message routed as appropriate.

How is the content filter specified?  Is this a multi-instance
configuration with "default_transport" set to the filter, or a single
instance configuration with "-o content_filter=" in the pre-filter
master.cf SMTP listener?

> Because no_address_mappings is set and the recipient address is in the
> virtual alias address class, however, the message instead gets
> rejected initially with the above mentioned error.

That is not what happens when "content_filter" is specified.

> Is this behavior intentional or just a side effect of how
> no_address_mappings is implemented?

    http://www.postfix.org/DEBUG_README.html#mail

Post:

   * postconf -nf
   * postconf -Mf
   * logs showing all processing of a rejected message

--
    Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Virtual alias address class and no_address_mappings

Peter Ajamian
On 30/12/19 5:15 pm, Viktor Dukhovni wrote:
> On Mon, Dec 30, 2019 at 04:37:32PM +1300, Peter wrote:
>
>> If someone uses virtual_address_domains and has
>> "receive_override_options = no_address_mappings", then postfix will
>> kick back an error of "User unknown in virtual alias table" for any
>> recipient in that domain.
>
> Addresses in virtual_alias_domains resolve to the error transport,
> unless rewritten to a "real domain", typically via virtual_alias_maps.

That makes sense.  I'm seeing the error transport in the logs.

> But, content_filter bypasses transport resolution, since all recipients
> go to the filter transport, so the below is not what typically happens.

But if that's the case then it shouldn't be going to the error transport.

>> User has a content filter configured which re-injects to postfix on
>> another port which does not have no_address_mappings set.  The intent
>> here is for address mappings to be ignored and the content filter run
>> initially then upon re-injection for the address mappings to be
>> processed and the message routed as appropriate.
>
> How is the content filter specified?  Is this a multi-instance
> configuration with "default_transport" set to the filter, or a single
> instance configuration with "-o content_filter=" in the pre-filter
> master.cf SMTP listener?

Config and logs linked below.

>> Because no_address_mappings is set and the recipient address is in the
>> virtual alias address class, however, the message instead gets
>> rejected initially with the above mentioned error.
>
> That is not what happens when "content_filter" is specified.

Then I got it wrong.  I tried to follow it in the source but got lost
very quickly and gave up.

>> Is this behavior intentional or just a side effect of how
>> no_address_mappings is implemented?
>
>      http://www.postfix.org/DEBUG_README.html#mail
>
> Post:

Ok, I needed to get permission from the OP to repost his info

>     * postconf -nf
>     * postconf -Mf

https://pastebin.com/raw/cfiTsNJD
(sorry pastebin.com was the ops choice but I gave the raw link at least)

>     * logs showing all processing of a rejected message

https://dpaste.org/FxmW/raw


Peter
Reply | Threaded
Open this post in threaded view
|

Re: Virtual alias address class and no_address_mappings

Viktor Dukhovni
On Mon, Dec 30, 2019 at 06:27:30PM +1300, Peter wrote:

> >      http://www.postfix.org/DEBUG_README.html#mail
> >
> >     * postconf -nf

    content_filter = scan:127.0.0.1:10025
    inet_interfaces = all
    receive_override_options = no_address_mappings
    virtual_alias_domains = [domain].com
    virtual_alias_maps = pgsql:/etc/postfix/pgsql_virtual_alias_maps.cf
        hash:/etc/postfix/virtual_alias_maps

With "ldap", "pgsql" and "mysql" it is generally a good idea to use
"proxy:ldap", "proxy:pgsql", ...

> >     * postconf -Mf

    smtp       inet  n       -       -       -       -       smtpd
    scan       unix  -       -       n       -       10      smtp
        -o smtp_send_xforward_command=yes
        -o disable_mime_output_conversion=yes
        -o smtp_generic_maps=
    localhost:10026 inet n   -       n       -       10      smtpd
        -o content_filter=
        -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
        -o smtpd_helo_restrictions=
        -o smtpd_client_restrictions=
        -o smtpd_sender_restrictions=
        -o smtpd_recipient_restrictions=permit_mynetworks,reject
        -o mynetworks=127.0.0.0/8
        -o smtpd_authorized_xforward_hosts=127.0.0.0/8

> >     * logs showing all processing of a rejected message

    Dec 30 00:59:13 li1145-60 postfix/smtpd[4934]: 84E583E814:
        client=mail-vs1-f52.google.com[209.85.217.52]
    Dec 30 00:59:13 li1145-60 postfix/cleanup[4939]: 84E583E814:
        message-id=<CAFbH14rfqLZo3=7WhZAUMxt9Yt3kyK=[hidden email]>
    Dec 30 00:59:13 li1145-60 postfix/qmgr[4761]: 84E583E814:
        from=<[hidden email]>, size=2491, nrcpt=1 (queue active)
    Dec 30 00:59:13 li1145-60 postfix/error[4940]: 84E583E814:
        to=<postmaster@[domain].com>, relay=none, delay=0.2, delays=0.2/0/0/0,
        dsn=5.1.1, status=bounced (User unknown in virtual alias table)
    Dec 30 00:59:13 li1145-60 postfix/bounce[4941]: 84E583E814:
        sender non-delivery notification: 8ECE03EA80
    Dec 30 00:59:13 li1145-60 postfix/qmgr[4761]: 84E583E814: removed

with "XFORWARD" the logs could be misleading, this could be the
processing of the message south of the content filter, and perhaps
the recipient is not listed in virtual_alias_maps.  Is there any
other logging for "84E583E814" or the message-id in question?

If the "content_filter" was set as reported, the message would
not have been handed off to the error transport.  The presence
of a filter:

    https://github.com/vdukhovni/postfix/blob/master/postfix/src/qmgr/qmgr_message.c#L1105-L1115

preëmpts recipient-specific resolution:

    https://github.com/vdukhovni/postfix/blob/master/postfix/src/qmgr/qmgr_message.c#L1121-L1127

that (for example) bounces unknown recipients in virtual alias domains.

The reported symptoms are not consistent with content_filter being set
for the message.

--
    Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Virtual alias address class and no_address_mappings

Peter Ajamian
On 30/12/19 8:42 pm, Viktor Dukhovni wrote:
> With "ldap", "pgsql" and "mysql" it is generally a good idea to use
> "proxy:ldap", "proxy:pgsql", ...

I agree, but in this particular case I was focusing on the problem at
hand.  I find on IRC it pays to not always get caught up in every
problem I see with someone's configuration if I want to see the end of them.

> with "XFORWARD" the logs could be misleading, this could be the
> processing of the message south of the content filter, and perhaps
> the recipient is not listed in virtual_alias_maps.

While this is possible I doubt it because he did mention that removing
no_address_mapping made the problem go away, so I think this is directly
related to that setting.  Also he insisted that the address was in the
db.  That said, it wouldn't be the first time I've seen someone say
things like that on IRC that turned out not to be true.

>  Is there any
> other logging for "84E583E814" or the message-id in question?

None that was provided, but I did reference the OP to this thread so
perhaps he will join in at this stage and provide more info.

> If the "content_filter" was set as reported, the message would
> not have been handed off to the error transport.  The presence
> of a filter:
>
>      https://github.com/vdukhovni/postfix/blob/master/postfix/src/qmgr/qmgr_message.c#L1105-L1115
>
> preëmpts recipient-specific resolution:
>
>      https://github.com/vdukhovni/postfix/blob/master/postfix/src/qmgr/qmgr_message.c#L1121-L1127
>
> that (for example) bounces unknown recipients in virtual alias domains.
>
> The reported symptoms are not consistent with content_filter being set
> for the message.

Yes, but does it preempt the resolution of whether a recipient exists at
all?  I thought that was done in smtpd, not qmgr.  I believe there might
be an implicit check_recipient_access on the end of
smtpd_recipient_restrictions that does that, but I'm not sure.  The idea
being that a message needs to be rejected before it hits the queue?

That said the logs do show the message going straight into qmgr, so I'm
really just baffled here.


Peter
Reply | Threaded
Open this post in threaded view
|

Re: Virtual alias address class and no_address_mappings

Viktor Dukhovni
On Mon, Dec 30, 2019 at 10:52:28PM +1300, Peter wrote:

> > The reported symptoms are not consistent with content_filter being set
> > for the message.
>
> Yes, but does it preempt the resolution of whether a recipient exists at
> all?  I thought that was done in smtpd, not qmgr.  I believe there might
> be an implicit check_recipient_access on the end of
> smtpd_recipient_restrictions that does that, but I'm not sure.

The existence checks in smtpd(8) are not affected by
receive_override_options.  The virtual alias tables are consulted as
appropriate.  In any case, the message was accepted.

In more sophisticated configurations, I've used different definitions of
virtual_alias_maps in smtpd(8) and cleanup(8), with the one used by
smtpd(8) used only for existence checks, avoiding expansion of LDAP
groups, ...  But since I wanted spam filtering (in particular a spam
quarantine) per-user, I did not use no_address_mappings.

> The idea being that a message needs to be rejected before it hits the
> queue?

This is why virtual_alias_maps is still used by smtpd(8) as part of
recipient validation.

> That said the logs do show the message going straight into qmgr, so I'm
> really just baffled here.

The reported symptoms are not consistent with the use of a
content_filter.

--
    Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Virtual alias address class and no_address_mappings

Stats Student
Thanks for looking into this and thank you Peter for reporting my
issue to the list.

To confirm, the alias is in the last database (
hash:/etc/postfix/virtual_alias_maps ) but the forwarding only works
if "receive_override_options = no_address_mappings" is NOT set
(commented out). And for the forwarding to actually happen the
content_filter must be disabled (unless there is a valid daemon
listening and re-injecting, etc). If no_address_mappings is enabled,
the message bounces with user unknown error.

virtual_alias_maps = pgsql:/etc/postfix/pgsql_virtual_alias_maps.cf
hash:/etc/postfix/virtual_alias_maps

Re: content_filter -- yes, I did uncomment this setting when
submitting the configuration to provide a fuller picture of my setup.
However, as you noted, the log entries were generated without the
content_filter set. Apologies for the confusion.

On Mon, Dec 30, 2019 at 8:51 AM Viktor Dukhovni
<[hidden email]> wrote:

>
> On Mon, Dec 30, 2019 at 10:52:28PM +1300, Peter wrote:
>
> > > The reported symptoms are not consistent with content_filter being set
> > > for the message.
> >
> > Yes, but does it preempt the resolution of whether a recipient exists at
> > all?  I thought that was done in smtpd, not qmgr.  I believe there might
> > be an implicit check_recipient_access on the end of
> > smtpd_recipient_restrictions that does that, but I'm not sure.
>
> The existence checks in smtpd(8) are not affected by
> receive_override_options.  The virtual alias tables are consulted as
> appropriate.  In any case, the message was accepted.
>
> In more sophisticated configurations, I've used different definitions of
> virtual_alias_maps in smtpd(8) and cleanup(8), with the one used by
> smtpd(8) used only for existence checks, avoiding expansion of LDAP
> groups, ...  But since I wanted spam filtering (in particular a spam
> quarantine) per-user, I did not use no_address_mappings.
>
> > The idea being that a message needs to be rejected before it hits the
> > queue?
>
> This is why virtual_alias_maps is still used by smtpd(8) as part of
> recipient validation.
>
> > That said the logs do show the message going straight into qmgr, so I'm
> > really just baffled here.
>
> The reported symptoms are not consistent with the use of a
> content_filter.
>
> --
>     Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Virtual alias address class and no_address_mappings

Viktor Dukhovni
On Mon, Dec 30, 2019 at 06:18:34PM -0800, Stats Student wrote:

> Re: content_filter -- yes, I did uncomment this setting when
> submitting the configuration to provide a fuller picture of my setup.
> However, as you noted, the log entries were generated without the
> content_filter set. Apologies for the confusion.

And of course "receive_override_options" MUST NOT be used when you don't
have a content_filter.  It is intended to make it possible to delay some
checks to post-filter processing, and exclude others already performed,
but absent a content_filter, it just breaks things by disabling required
some processing.

Please NEVER report configuration settings that are different from those
that were in effect when the reported logs were generated.  That wastes
the time of people trying to help you, and is simply rude, however well
intentioned (that "fuller picture" was anything but).

--
    Viktor.