Virtual mapping without classifying

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Virtual mapping without classifying

ewilkison
Virtual mapping without classifying
We're working on a project that will use Postfix as gateway servers from the
Internet to a couple a couple of internal mail systems.  We've got it setup
with hash tables to map about 30K virtual users to their destination
address.  Mail to non existing users should be rejected.
One of the requirements is to map also map several countries domains back to
the main .com domain.  For example, [hidden email] can also receive mail as
[hidden email], [hidden email], etc.  To accomplish this we used regex
virtual mapping:
/(.*)@example.uk/ $[hidden email]
/(.*)@example.in/ $[hidden email]

This system works well for known users, but does not reject mail for non
existing country domain users. For example [hidden email] is accepted
then a bounce message is generated because there is no virtual mapping for
[hidden email]. When [hidden email] is mapped to [hidden email]
via the regex mapping it is also classified as an authorized destination. 
Is there a way to have postfix perform this first level of mapping without
classifying the recipient as authorized? 

Is there a better way to work around this issue?




--
Sent from: http://postfix.1071664.n5.nabble.com/Postfix-Users-f2.html
Reply | Threaded
Open this post in threaded view
|

Re: Virtual mapping without classifying

Wietse Venema
ewilkison:

> Virtual mapping without classifying
> We're working on a project that will use Postfix as gateway servers from the
> Internet to a couple a couple of internal mail systems.? We've got it setup
> with hash tables to map about 30K virtual users to their destination
> address.? Mail to non existing users should be rejected.
> One of the requirements is to map also map several countries?domains back to
> the main .com domain.? For example, [hidden email] can also receive mail as
> [hidden email], [hidden email], etc.? To accomplish this we used regex
> virtual mapping:
> /(.*)@example.uk/ $[hidden email]
> /(.*)@example.in/ $[hidden email]
>
> This system works well for known users, but does not reject mail for non
> existing?country domain users. For example [hidden email] is accepted
> then a bounce message is generated because there is no virtual mapping for
> [hidden email]. When [hidden email] is mapped to [hidden email]
> via the regex mapping it is also classified as an authorized destination.?
> Is there a way to have postfix perform this first level of mapping without
> classifying the recipient?as authorized??

No. The virtual alias map does not verify that the user really exists.

> Is there a better way to work around this issue?

1) Periodically, populate your virtual aliases with real data.

2) Use reject_unverified_recipient, as described in
http://www.postfix.org/ADDRESS_VERIFICATION_README.html#recipient

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Virtual mapping without classifying

ewilkison
Thank you for the input.  

Our virtual alias files are script generated and updated periodically.  I had considered adding a map for each user for each of the domains.  However, we have about 80 domains.  That times our 30k users and we'd have almost 2.5 million aliases. Seems like a very large list that could cause performance issues under load.   Would you agree?

I had also considered using validation, but one of the mail systems we are delivering to is behind yet another relay that does not reject unknown users.  We could potentially  set up a validation transport to work around it, but internal political concerns would make that solution difficult.  

Doing more research yesterday I came up with another possible solution I'd like to get your opinion of.  What do you think of using smtpd_command_filter to dynamically update the recipients. Something like:

/^RCPT\s+TO:\s*(.*)@example\.\b(in|cn|us)\b(>*)/ RCPT TO:$[hidden email]$3

This seems to work form my initial testing.  Would there be any issues that I'm not seeing using this solution?  Any corner cases that the regex would not handle well?


From: [hidden email] <[hidden email]> on behalf of Wietse Venema <[hidden email]>
Sent: Tuesday, February 18, 2020 12:59 PM
To: Postfix users <[hidden email]>
Subject: [External] Re: Virtual mapping without classifying
 
ewilkison:
> Virtual mapping without classifying
> We're working on a project that will use Postfix as gateway servers from the
> Internet to a couple a couple of internal mail systems.? We've got it setup
> with hash tables to map about 30K virtual users to their destination
> address.? Mail to non existing users should be rejected.
> One of the requirements is to map also map several countries?domains back to
> the main .com domain.? For example, [hidden email] can also receive mail as
> [hidden email], [hidden email], etc.? To accomplish this we used regex
> virtual mapping:
> /(.*)@example.uk/ $[hidden email]
> /(.*)@example.in/ $[hidden email]
>
> This system works well for known users, but does not reject mail for non
> existing?country domain users. For example [hidden email] is accepted
> then a bounce message is generated because there is no virtual mapping for
> [hidden email]. When [hidden email] is mapped to [hidden email]
> via the regex mapping it is also classified as an authorized destination.?
> Is there a way to have postfix perform this first level of mapping without
> classifying the recipient?as authorized??

No. The virtual alias map does not verify that the user really exists.

> Is there a better way to work around this issue?

1) Periodically, populate your virtual aliases with real data.

2) Use reject_unverified_recipient, as described in
http://www.postfix.org/ADDRESS_VERIFICATION_README.html#recipient

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Virtual mapping without classifying

Wietse Venema
Eric Wilkison:
> Thank you for the input.
>
> Our virtual alias files are script generated and updated periodically.
> I had considered adding a map for each user for each of the domains.
> However, we have about 80 domains.  That times our 30k users and
> we'd have almost 2.5 million aliases. Seems like a very large list
> that could cause performance issues under load.   Would you agree?

That depends on how one does it. But, in order to accept mail for
valid recipients, the perimeter Postfix MTA needs to know what is
valid, or have a way to discover what is valid, such as
reject_unverified_recipient.

> I had also considered using validation, but one of the mail systems
> we are delivering to is behind yet another relay that does not
> reject unknown users.  We could potentially  set up a validation
> transport to work around it, but internal political concerns would
> make that solution difficult.

Too bad, I can't help you with the non-technical issues.

> Doing more research yesterday I came up with another possible
> solution I'd like to get your opinion of.  What do you think of
> using smtpd_command_filter to dynamically update the recipients.
> Something like:
>
> /^RCPT\s+TO:\s*(.*)@example\.\b(in|cn|us)\b(>*)/ RCPT TO:$[hidden email]$3

Assuming that this will block all invalid addresses, a minor downside
I see is that any delivery errors will report the modified recipient
address.

One cannot simply append an ORCPT=whatever parameter to the RCPT TO
command, because RFC 3461 requires that the ORCPT value be properly
encoded, and I cannot recommend doing that with regular expressions.

> This seems to work form my initial testing.  Would there be any
> issues that I'm not seeing using this solution?  Any corner cases
> that the regex would not handle well?

If every valid [hidden email] is also valid as
[hidden email], then that would work.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Virtual mapping without classifying

Viktor Dukhovni
In reply to this post by ewilkison
On Wed, Feb 19, 2020 at 10:02:46PM +0000, Eric Wilkison wrote:

> Our virtual alias files are script generated and updated periodically.
> I had considered adding a map for each user for each of the domains.
> However, we have about 80 domains.  That times our 30k users and we'd
> have almost 2.5 million aliases. Seems like a very large list that
> could cause performance issues under load.   Would you agree?

No.  You're not likely to see any significant performance difference.
That's why database tables are indexed.  Yes, there's a small
logarithmic cost factor, but it won't matter in practice.

But I also recommend strongly against aliasing every user into every
variant domain.  That just buys your users multiple copies of every spam
message, because spammers will treat each address as a separate
recipient.

Instead, allow users to opt-in to additional domains as needed,
and maintain a list of actual registered aliases, rather than
a machine generated cross-product.

> Doing more research yesterday I came up with another possible solution
> I'd like to get your opinion of.  What do you think of using
> smtpd_command_filter to dynamically update the recipients.

A gross hack.  Don't go there.

--
    Viktor.