Virtual canonical domains?

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Virtual canonical domains?

Ville Walveranta
The exciting quest to Postfix continues...

Tonight's question is about virtual canonical domains (the term is
obviously coined by me since there are no Google hits with it before
this post gets indexed ;). I have currently (for clarity's sake while
I'm learning the system) defined virtual_mailbox_domains,
virtual_mailbox_maps, virtual_alias_domains, and _virtual_alias_maps
as individual files:

The mentioned domains are obviously internal..

#virtual_alias_domains:
thirddomain.com         20081114
fourthdomain.com       20081115

#virtual_mailbox_domains:
someotherdomain.com     20081115

#virtual_alias_maps:
[hidden email]   [hidden email], [hidden email]
[hidden email]      [hidden email]
@thirddomain.com                        [hidden email]
postmaster                                   [hidden email]

#virtual_mailbox_maps:
[hidden email]        someotherdomain.com/info/
[hidden email]      someotherdomain.com/test1/
[hidden email]      someotherdomain.com/test2/
[hidden email]      someotherdomain.com/test3/

--

What I'd like to accomplish is to create a new virtual domain that
mirrors the logins of an existing domain without a catch-all. In my
example above the domain fourthdomain.com has been defined as a
virtual alias domain, but how do I enable it to receive email for the
exact same email accounts as those of someotherdomain.com (i.e.
"[hidden email]", "[hidden email]",
"[hidden email]", "test1fourthdomain.com",
"[hidden email]", and "[hidden email]")? Perhaps I
have to create all corresponding aliases for fourthdomain.com? Or is
there a simpler way? In the production environment I'm working on to
set up there are about a dozen domains with about 20 identical
accounts so that a given username receives email with all those dozen
domain names. If I can create virtual canonical domains, I could
create the accounts once and alias the rest of the domains as opposed
to creating 20 accounts and about 220 redirect aliases.

Perhaps this could be accomplished with check_recipient_access using
PCRE table that would have a IF.. REDIRECT ..ENDIF clause for each
address... or maybe there's a really simple way to do it (for why I
ask before I try anything else). I'm thinking along the lines of the
"virtualdomains" control file in qmail where it is possible assign
multiple domain names to a mailbox user which in turn automatically
aliases all 'attached' domains to the virtual user accounts defined
for that virtual domain.

Thank you all for help, again!! :)

Ville
Reply | Threaded
Open this post in threaded view
|

Re: Virtual canonical domains?

mouss-2
Ville Walveranta wrote:

> The exciting quest to Postfix continues...
>
> Tonight's question is about virtual canonical domains (the term is
> obviously coined by me since there are no Google hits with it before
> this post gets indexed ;). I have currently (for clarity's sake while
> I'm learning the system) defined virtual_mailbox_domains,
> virtual_mailbox_maps, virtual_alias_domains, and _virtual_alias_maps
> as individual files:
>
> The mentioned domains are obviously internal..
>
> #virtual_alias_domains:
> thirddomain.com         20081114
> fourthdomain.com       20081115
>
> #virtual_mailbox_domains:
> someotherdomain.com     20081115
>
> #virtual_alias_maps:
> [hidden email]   [hidden email], [hidden email]
> [hidden email]      [hidden email]
> @thirddomain.com                        [hidden email]
> postmaster                                   [hidden email]
>
> #virtual_mailbox_maps:
> [hidden email]        someotherdomain.com/info/
> [hidden email]      someotherdomain.com/test1/
> [hidden email]      someotherdomain.com/test2/
> [hidden email]      someotherdomain.com/test3/
>
> --
>
> What I'd like to accomplish is to create a new virtual domain that
> mirrors the logins of an existing domain without a catch-all. In my
> example above the domain fourthdomain.com has been defined as a
> virtual alias domain, but how do I enable it to receive email for the
> exact same email accounts as those of someotherdomain.com (i.e.
> "[hidden email]", "[hidden email]",
> "[hidden email]", "test1fourthdomain.com",
> "[hidden email]", and "[hidden email]")? Perhaps I
> have to create all corresponding aliases for fourthdomain.com? Or is
> there a simpler way? In the production environment I'm working on to
> set up there are about a dozen domains with about 20 identical
> accounts so that a given username receives email with all those dozen
> domain names. If I can create virtual canonical domains, I could
> create the accounts once and alias the rest of the domains as opposed
> to creating 20 accounts and about 220 redirect aliases.
>
> Perhaps this could be accomplished with check_recipient_access using
> PCRE table that would have a IF.. REDIRECT ..ENDIF clause for each
> address... or maybe there's a really simple way to do it (for why I
> ask before I try anything else). I'm thinking along the lines of the
> "virtualdomains" control file in qmail where it is possible assign
> multiple domain names to a mailbox user which in turn automatically
> aliases all 'attached' domains to the virtual user accounts defined
> for that virtual domain.
>


in theory, you could use wildcard virtual_alias_maps:
@alternatename.example @primary.example

unfortunately, this makes all addresses *@alternatename.example valid
during the smtp transaction, and this will cause a bounce if the address
cannot be delivered (if [hidden email] does not exist). Thus
backscatter.

so the choices I can see are:

- use a script to create the virtual aliases

- if your users are stored in sql, you could use sql statements to
generate the aliases (only when the "primary" address really exists)

- write a policy service to do recipient validation. Then you can use
wildcard virtual aliases.

...
Reply | Threaded
Open this post in threaded view
|

Re: Virtual canonical domains?

Ville Walveranta
On Sun, Nov 16, 2008 at 1:11 AM, mouss <[hidden email]> wrote:
> in theory, you could use wildcard virtual_alias_maps:
> @alternatename.example  @primary.example
>
> unfortunately, this makes all addresses *@alternatename.example valid during
> the smtp transaction, and this will cause a bounce if the address cannot be
> delivered (if [hidden email] does not exist). Thus backscatter.

Yeah, I probably wouldn't want to do that..

> so the choices I can see are:
>
> - use a script to create the virtual aliases
>
> - if your users are stored in sql, you could use sql statements to generate
> the aliases (only when the "primary" address really exists)
>
> - write a policy service to do recipient validation. Then you can use
> wildcard virtual aliases.

I'm undecided on the back-end as of yet; I'd like to use LDAP for
authentication, possibly against AD (as was discussed here earlier).
But some other data such as alias maps would seem to be more
straightforward to store in SQL. Someplace I've seen a warning against
mixing LDAP and SQL — may have been in the "Book of Postfix", but
since MySQL and OpenLDAP are on the server anyway, I don't see why I
couldn't use them both. I'll look into a policy service option, too.

I'll probably end up using SQL for the alias maps with some kind of
simple front-end (quickly put together with CodeCharge) to maintain
them. Perhaps "Virtual Canonical Domains" is something Wietse could
address in a future version of Postfix. I think lack of such option is
the first thing I've come across that qmail has on Postfix.

Ville
Reply | Threaded
Open this post in threaded view
|

Re: Virtual canonical domains?

mouss-2
Ville Walveranta a écrit :
>
> I'm undecided on the back-end as of yet; I'd like to use LDAP for
> authentication, possibly against AD (as was discussed here earlier).
> But some other data such as alias maps would seem to be more
> straightforward to store in SQL. Someplace I've seen a warning against
> mixing LDAP and SQL — may have been in the "Book of Postfix", but
> since MySQL and OpenLDAP are on the server anyway, I don't see why I
> couldn't use them both. I'll look into a policy service option, too.
>

I don't remember any such warning, and I don't see what problem this
would cause even if the servers are on different machines.

> I'll probably end up using SQL for the alias maps with some kind of
> simple front-end (quickly put together with CodeCharge) to maintain
> them. Perhaps "Virtual Canonical Domains" is something Wietse could
> address in a future version of Postfix. I think lack of such option is
> the first thing I've come across that qmail has on Postfix.

I don't know what you mean by "virtual canonical domains". These are
"virtual alias domains".

The problem you are facing is that virtual aliases are not recursively
expanded at smtp time.

if you don't need recursion, you can use sql easily. otherwise, a policy
server can do whatever you want.

Reply | Threaded
Open this post in threaded view
|

Re: Virtual canonical domains?

Ville Walveranta
On Mon, Nov 17, 2008 at 1:28 AM, mouss <[hidden email]> wrote:
>> straightforward to store in SQL. Someplace I've seen a warning against
>> mixing LDAP and SQL — may have been in the "Book of Postfix", but
>  I don't remember any such warning, and I don't see what problem this
>  would cause even if the servers are on different machines.

Ah, it was on "Linuxtopia" website (http://tinyurl.com/564q7r). In
other words, it was more of someone's opinion than by any means a
recommendation from an "authoritative source".

> I don't know what you mean by "virtual canonical domains". These are
> "virtual alias domains".

It's simply a virtual alias domain that aliases all users from another
virtual domain. (Now Google search finds this thread and nothing else
if you search for "virtual canonical domains" with quotes ;-).

> The problem you are facing is that virtual aliases are not recursively
> expanded at smtp time.

Exactly.

> if you don't need recursion, you can use sql easily. otherwise, a policy
> server can do whatever you want.

The policy server looks really interesting! I'll give it a try, and
also see if Darren's suggestion would work for this particular issue.

Ville
Reply | Threaded
Open this post in threaded view
|

Re: Virtual canonical domains?

Ville Walveranta
In reply to this post by Ville Walveranta
On Mon, Nov 17, 2008 at 1:04 AM, Darren Pilgrim <[hidden email]> wrote:
> Use a pcre map to return the local part @someotherdomain.com:
>
> /^(.+)@fourthdomain\.com$/ ${1}@someotherdomain.com

Where would you put that pcre map? I tried few different patterns in
check_recipient_access in smtpd_recipient_restrictions but Postfix
didn't like it ("server misconfiguration").

Ville
Reply | Threaded
Open this post in threaded view
|

Re: Virtual canonical domains?

mouss-2
Ville Walveranta a écrit :
> On Mon, Nov 17, 2008 at 1:04 AM, Darren Pilgrim <[hidden email]> wrote:
>> Use a pcre map to return the local part @someotherdomain.com:
>>
>> /^(.+)@fourthdomain\.com$/ ${1}@someotherdomain.com
>
> Where would you put that pcre map?

depends on what you want to do with the map. but don't use this in
virtual_alias_maps, because
1- as said before, it rbeaks recipient validation,
2- you don't need regular expressions here:
@fourthdomain.com @someotherdomain.com

does exactly the same.

 I tried few different patterns in
> check_recipient_access in smtpd_recipient_restrictions but Postfix
> didn't like it ("server misconfiguration").
>

without more infos, we can't help. you first ned to verify that your
postfix was built with pcre support:
# postconf -m
and if so, you can use pcre maps in many places.

if you get an error, look at other errors/warnings in the logs and
you'll see the reason for the error.
Reply | Threaded
Open this post in threaded view
|

Re: Virtual canonical domains?

Ville Walveranta
In reply to this post by Ville Walveranta
ACL Policy Daemon for Postfix might do the trick without me having to write the policy daemon myself. It provides numerous ACL methods and Regex ACLs.

Ville


Reply | Threaded
Open this post in threaded view
|

Re: Virtual canonical domains?

Ville Walveranta
In reply to this post by Ville Walveranta
On Tue, Nov 18, 2008 at 11:43 AM, Darren Pilgrim <[hidden email]> wrote:
/^(info|sales|test1)@fourthdomain\.com$/ ${1}@someotherdomain.com


I'm not having luck with that.

I put...

[hidden email]       [hidden email]
[hidden email]       [hidden email]
/^(user1|user2)@fourthdomain\.com$/ ${1}@someotherdomain.com

.. in virtual_alias_maps and refreshed with postmap, yet I get..

RCPT TO:<[hidden email]>
450 4.1.1 <[hidden email]>: Recipient address rejected:
undeliverable address: User unknown in virtual alias table

---
current main.cf:

## DELTAS TO MAIN.CF.DEFAULT
##
## For the syntax, and for a complete parameter list,
## see the postconf(5) manual page ("man 5 postconf"),
## or see http://www.postfix.org/postconf.5.html

#soft_bounce = no
debug_peer_level = 9
debug_peer_list = 127.0.0.1

data_directory = /var/db/postfix
queue_directory = /var/spool/postfix
command_directory = /usr/local/sbin
daemon_directory = /usr/local/libexec/postfix
manpage_directory = /usr/local/man
sendmail_path = /usr/local/sbin/sendmail
newaliases_path = /usr/local/bin/newaliases
mailq_path = /usr/local/bin/mailq
readme_directory = $config_directory/README_FILES
sample_directory = /usr/local/etc/postfix
html_directory = no

mail_owner = postfix
setgid_group = maildrop

myhostname = myhostname.com
mydomain = mydomain.myhostname.com
myorigin = $myhostname

mydestination =
        $myhostname
        localhost.$mydomain
        localhost
        mydomain.myhostname.com

mynetworks_style = host
mynetworks = 192.168.1.0/24
relay_domains = $mydestination
#delay_warning_time = 4h

# define here the listening interfaces
# that do _not_ have custom rules
inet_interfaces = 127.0.0.1, 192.168.1.99

# execute `postsuper -r ALL' & reload if you disable content_filter!
content_filter = scan:[127.0.0.1]:10025
receive_override_options = no_address_mappings

smtpd_helo_required = yes
smtpd_sasl_auth_enable = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth-client
broken_sasl_auth_clients = yes
disable_vrfy_command = yes

dovecot_destination_recipient_limit = 1
mailbox_transport = dovecot
virtual_transport = dovecot
virtual_mailbox_base = /home/vmail
virtual_mailbox_domains = $config_directory/tables/virtual_mailbox_domains
virtual_mailbox_maps = hash:$config_directory/tables/virtual_mailbox_maps
virtual_alias_domains = $config_directory/tables/virtual_alias_domains
virtual_alias_maps = hash:$config_directory/tables/virtual_alias_maps
virtual_uid_maps = static:2000
virtual_gid_maps = static:2000

smtpd_client_restrictions =
        permit_mynetworks
        permit_inet_interfaces
        reject

smtpd_client_restrictions_katharion =
        permit_mynetworks
        permit_sasl_authenticated
        check_client_access
hash:$config_directory/tables/smtpd_client_access_katharion
        reject

smtpd_helo_restrictions =
        reject_invalid_helo_hostname
        reject_non_fqdn_helo_hostname
        permit_mynetworks
        permit_sasl_authenticated
        reject_unknown_helo_hostname

smtpd_etrn_restrictions =
        permit_mynetworks
        reject

smtpd_recipient_restrictions =
        reject_non_fqdn_recipient
        reject_non_fqdn_sender
        reject_unknown_sender_domain
        reject_unknown_recipient_domain
        reject_unverified_recipient
        check_recipient_access
pcre:$config_directory/tables/smtpd_recipient_access
        permit_mynetworks
        permit_sasl_authenticated
        reject_non_fqdn_hostname
        reject_invalid_hostname
        reject_unauth_destination

smtpd_recipient_restrictions_katharion =
        reject_non_fqdn_recipient
        reject_non_fqdn_sender
        reject_unknown_sender_domain
        reject_unknown_recipient_domain
        reject_unverified_recipient
        check_recipient_access
pcre:$config_directory/tables/smtpd_recipient_access_katharion
        permit_mynetworks
        permit_sasl_authenticated
        reject_non_fqdn_hostname
        reject_invalid_hostname
        reject_unauth_destination

smtpd_data_restrictions =
        reject_multi_recipient_bounce
        reject_unauth_pipelining
Reply | Threaded
Open this post in threaded view
|

Re: Virtual canonical domains?

mouss-2
Ville Walveranta a écrit :

> On Tue, Nov 18, 2008 at 11:43 AM, Darren Pilgrim <[hidden email]> wrote:
> /^(info|sales|test1)@fourthdomain\.com$/ ${1}@someotherdomain.com
>
>
> I'm not having luck with that.
>
> I put...
>
> [hidden email]       [hidden email]
> [hidden email]       [hidden email]
> /^(user1|user2)@fourthdomain\.com$/ ${1}@someotherdomain.com
>
> .. in virtual_alias_maps and refreshed with postmap, yet I get..

you are mixing different formats.

virtual_alias_maps =
        hash:/etc/postfix/virtual_alias.hash
        pcre:/etc/postfix/virtual_alias.pcre

then put the two first lines in the .hash file and the last one (the
/.../ ... line) in the .pcre file. and by the way, only postmap the
.hash file.

> RCPT TO:<[hidden email]>
> 450 4.1.1 <[hidden email]>: Recipient address rejected:
> undeliverable address: User unknown in virtual alias table
>
> ---
> current main.cf:
>

in the future, send output of 'postconf -n' instead of main.cf.

> [snip]
Reply | Threaded
Open this post in threaded view
|

Re: Virtual canonical domains?

Ville Walveranta
In reply to this post by Ville Walveranta
On Tue, Nov 18, 2008 at 12:25 PM, Darren Pilgrim <[hidden email]> wrote:
> You have different lookup types in the same table.  The pcre line goes in a
> second lookup table (i.e., virtual_alias_maps.pcre) added to
> virtual_alias_maps:
>
> virtual_alias_maps =
>        hash:${config_directory}/tables/virtual_alias_maps
>        pcre:${config_directory}/tables/virtual_alias_maps.pcre
>

That works! The domains in question have about 20 users that would
need to be aliased through a dozen or so domains. So while not exactly
pretty and probably not terribly powerful, it'll save the typing. I
can have 20 of these:

/^(user1|user2|user3|user4|user5|user6|user7|user8|user9|user10|user11|user12|user13|user14|user15|user16|user17|user18|user19|user20)@aliasdomain1\.com$/
${1}@targetdomain.com

.. instead of 240 individual alias lines (and since the users are the
same for all 20 domains, just the "aliasdomain" name needs to be
modified on each line).

ACL through policy daemon or some sort of SQL setup may be the
ultimate solution but this will work well for starters.

Thank you very much for helping me out with this!

Ville
Reply | Threaded
Open this post in threaded view
|

Re: Virtual canonical domains?

mouss-2
Ville Walveranta a écrit :

> On Tue, Nov 18, 2008 at 12:25 PM, Darren Pilgrim <[hidden email]> wrote:
>> You have different lookup types in the same table.  The pcre line goes in a
>> second lookup table (i.e., virtual_alias_maps.pcre) added to
>> virtual_alias_maps:
>>
>> virtual_alias_maps =
>>        hash:${config_directory}/tables/virtual_alias_maps
>>        pcre:${config_directory}/tables/virtual_alias_maps.pcre
>>
>
> That works! The domains in question have about 20 users that would
> need to be aliased through a dozen or so domains. So while not exactly
> pretty and probably not terribly powerful, it'll save the typing. I
> can have 20 of these:
>
> /^(user1|user2|user3|user4|user5|user6|user7|user8|user9|user10|user11|user12|user13|user14|user15|user16|user17|user18|user19|user20)@aliasdomain1\.com$/
> ${1}@targetdomain.com
>
> .. instead of 240 individual alias lines (and since the users are the
> same for all 20 domains, just the "aliasdomain" name needs to be
> modified on each line).
>

after some time, a script will save more...

# cat alias-target.users
user1
user2
...
# cat myscript
#!/bin/sh
grep -v "^#" alias-target.users | while read _user; do
  echo "${_user}@alias.example  ${_user}@target.example"
done



> ACL through policy daemon or some sort of SQL setup may be the
> ultimate solution but this will work well for starters.
>


Reply | Threaded
Open this post in threaded view
|

Re: Virtual canonical domains?

Ville Walveranta
On Tue, Nov 18, 2008 at 1:25 PM, mouss <[hidden email]> wrote:

> after some time, a script will save more...
>
> # cat alias-target.users
> user1
> user2
> ...
> # cat myscript
> #!/bin/sh
> grep -v "^#" alias-target.users | while read _user; do
>  echo "${_user}@alias.example  ${_user}@target.example"
> done

From the looks of that the "myscript" can be then referenced from
virtual_alias_maps in main.cf. Yes, having to enter the user list just
once would be even better (easier to maintain, etc.). I'll give it a
try in the morning.

Ville
Reply | Threaded
Open this post in threaded view
|

Re: Virtual canonical domains?

Noel Jones-2
Ville Walveranta wrote:

> On Tue, Nov 18, 2008 at 1:25 PM, mouss <[hidden email]> wrote:
>> after some time, a script will save more...
>>
>> # cat alias-target.users
>> user1
>> user2
>> ...
>> # cat myscript
>> #!/bin/sh
>> grep -v "^#" alias-target.users | while read _user; do
>>  echo "${_user}@alias.example  ${_user}@target.example"
>> done
>
> From the looks of that the "myscript" can be then referenced from
> virtual_alias_maps in main.cf. Yes, having to enter the user list just
> once would be even better (easier to maintain, etc.). I'll give it a
> try in the morning.
>
> Ville

No, "myscript" is outside of postfix and run as needed.
Shell syntax is not valid inside virtual_alias_maps (or any
postfix map).

I would put this in a Makefile so I could just run "make" and
have it rebuild what needs rebuilding.  Here's an example:
http://www.postfix.org/DATABASE_README.html#safe_db

--
Noel Jones