Problems with lmtp

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Problems with lmtp

Doug Barton
[ Trying this again as I think I sent to the wrong address the first time ]

Greetings,

First I'd like to say thank you to all of the developers and contributors to Postfix. I've used it for many years with great success and happiness. :)

I'm on Ubuntu Server 16.04 (up to date) and using the stock postfix package (3.10-3).

I have a working installation with postfix and dovecot, and I want to add sieve to it, so I am trying to configure postfix to use lmtp for its delivery service. However it is ignoring that request, and for every message I get "status=sent (delivered to maildir)" and it shows up in my Inbox.

On my mail host I have 1 normal user. I have postfix configured to accept mail for several different domains, and each domain has a lot of different mail usernames (I use this for mailing lists and such). I use the virtual_maps feature of postfix, and have a map file that looks like this:

[hidden email] normaluser
[hidden email] normaluser
[hidden email] normaluser
...

All of this works great, and mail for all the different usernames and domains gets delivered into my one real user's Maildir, and I can see the mail with my IMAP clients.

I've configured sieve in dovecot, and I can see the socket for lmtp in /var/spool/postfix/private/.

So according to all the tutorials I've read my assumption is that my next step is this in postfix' main.cf:

virtual_transport = lmtp:unix:private/dovecot-lmtp

which I did, and postfix restarts with no errors. But, it seems to avoid lmtp altogether, and it delivers straight to my Maildir Inbox every time.

I have since learned that I probably don't want virtual_transport for this, but I probably do want local_transport. The problem is that if I put in local_transport = lmtp:unix:private/dovecot-lmtp I get a bounce every time:

Mar 15 18:01:20 dougbarton postfix/lmtp[11793]: 8BCD38F: to=<[hidden email]>, relay=dougbarton.us[private/dovecot-lmtp], delay=0.03, delays=0.01/0/0/0.01, dsn=5.1.1, status=bounced (host dougbarton.us[private/dovecot-lmtp] said: 550 5.1.1 <[hidden email]> User doesn't exist: [hidden email] (in reply to RCPT TO command))

From further reading it seems that I need to add some sort of additional mapping, but it's not clear to me what. Adding my virtual_maps file to local_recipient_maps didn't work. I also tried 'local_recipient_maps = ' to see if I could rule out a chroot issue, but that didn't work either.

I tried adding logging to the lmtp process, and the only thing I get in the log is this:
Mar 15 17:57:57 lmtp(11397): Info: Connect from local
Mar 15 17:57:57 lmtp(11397): Info: Disconnect from local: Successful quit

That happens twice per delivery attempt.

I've been working on this for two days, and I'm probably missing something really obvious, but I would appreciate your assistance. Testing has been difficult because the messages bounce hard and I get a lot of mail every day.

Here is postconf -n with security-related and boring items removed.

alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
compatibility_level = 2
home_mailbox = Maildir/
local_recipient_maps = proxy:unix:passwd.byname $alias_maps
milter_default_action = accept
mydestination = $mydomain, localhost.$mydomain, localhost
mydomain = dougbarton.us
myhostname = dougbarton.us
smtpd_delay_reject = yes
smtpd_helo_required = yes
smtpd_helo_restrictions = permit_mynetworks permit_sasl_authenticated reject_non_fqdn_helo_hostname reject_invalid_helo_hostname permit
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_pipelining reject_unauth_destination reject_unverified_recipient reject_rbl_client zen.spamhaus.org check_policy_service inet:127.0.0.1:10023 check_policy_service unix:private/spf-policy
smtpd_sender_restrictions = permit_mynetworks permit_sasl_authenticated reject_non_fqdn_sender reject_unknown_sender_domain permit
unverified_recipient_reject_code = 550
virtual_maps = hash:/etc/postfix/virtual_addresses
virtual_transport = lmtp:unix:private/dovecot-lmtp

I'm sorry that this message is so long, but apparently my situation is somewhat unusual (I wasn't able to find any similar configurations after a lot of searching) and I wasn't sure what to include.

Any help will be greatly appreciated.

Doug
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lmtp

Viktor Dukhovni
On Thu, Mar 16, 2017 at 04:02:58AM +0000, Doug wrote:

>      $ postconf -d mail_version
>  
> Yes, 3.1.0, thank you.

Cool.  I would expect that this likely contains backports of later
patches, but unfortunately the Linux distros tend to avoid backporting
upstream version number updates, so it is difficult to tell whether
you have all the fixes from 3.1.0 to 3.1.4, but it is quite possible
that you do.

> The offending record is gone.

Thanks.

> given the staggeringly low rate of adoption for DANE I'm not missing
> anything at this point.

Well, DANE is doing comparatively fine, (~111000 domains and
growing, this takes time).

> Meanwhile, I would welcome your help with my lmtp issue, or not, as you
> see fit. I do expect though that this is the last I hear from you on the
> TLSA topic, as your private bullying was quite tedious enough already.

I am sorry you feel that way, I did not expect that ten or so emails
over two weeks back in 2014 as part of a discussion with the draft
coauthors would amount to bullying.  Anyway, that's all water under
the bridge.

As promised, I'll help with the LDAP issue:

> On my mail host I have 1 normal user. I have postfix configured to accept
> mail for several different domains, and each domain has a lot of different
> mail usernames (I use this for mailing lists and such). I use the
> virtual_maps feature of postfix, and have a map file that looks like this:

FWIW, the modern name for this feature is "virtual_alias_maps",
this rewrites all envelope recipients (regardless of address class,
see http://www.postfix.org/ADDRESS_CLASS_README.html) on input in
the cleanup(8) service.  By the time mail enters the queue recipients
are already rewritten as specified in that table.

> All of this works great, and mail for all the different usernames and
> domains gets delivered into my one real user's Maildir, and I can see the
> mail with my IMAP clients.

The important thing to understand here is the difference between
the "local", "virtual alias" and "virtual mailbox" address classes,
as explained in ADDRESS_CLASS_README.

    - Domains listed in "mydestination" are "local" and designed for
      delivery of mail to mailbox that belong to local unix accounts.
      These are delivered via $local_transport, which is almost
      always "local" (the local(8) delivery agent (LDA) in master.cf)
      or "error" if local delivery is disabled.

      When local delivery is not disabled, the local(8) delivery
      agent does further rewriting as specified in $alias_maps,
      and then by default delivers to the user's mbox.  However,
      local(8) is quite fancy, and can also deliver to commands
      via mailbox_command (often procmail), to mailboxes in a
      home directory (via $home_mailbox) or to an alternative
      transport (via mailbox_transport).  There are more options
      see http://www.postfix.org/local.8.html for detail.

    - Domains listed in virtual_alias_domains are exclusively
      designated as holding only aliases to other real domains.
      Don't make the mistake of assuming that a domain must
      be listed here in order for virtual_alias_maps to happen.
      See above.

    - Domains listed in virtual_mailbox_domains are for hosting
      "virtual users", who do not necessarily have user login
      accounts.  These are handled by $virtual_transport, which
      defaults to "virtual", the Postfix built-in VDA, but other
      VDAs can substituted as desired.

As explained in ADDRESS_CLASS_README, each address class comes
with its recipient validation table:

    - local: $local_recipient_maps
    - virtual alias: $virtual_alias_maps (implicitly used for all classes,
      in addition to the class-specific validation tables).
    - virtual mailbox: $virtual_mailbox_maps


> So according to all the tutorials I've read my assumption is that my next
> step is this in postfix' main.cf:
>
> virtual_transport = lmtp:unix:private/dovecot-lmtp

The reason this failed for you, is that your domains are "local"
(listed in mydestination) so their delivery is controlled via
$local_transport not $virtual_transport.

> I have since learned that I probably don't want virtual_transport for
> this, but I probably do want local_transport. The problem is that if I
> put in local_transport = lmtp:unix:private/dovecot-lmtp I get a bounce
> every time:

The reason this may fail is that mail is no longer subjected to
$alias_maps when you do that.  Of course another reason might be
that dovecot is not fully configured to receive mail for the
addresses in question.  In the former case, you can still
get aliases(5) to be processed by using:

    $mailbox_transport = lmtp:unix:private/dovecot-lmtp

instead of replacing $local_transport.

> Mar 15 18:01:20 dougbarton postfix/lmtp[11793]: 8BCD38F:
> to=<[hidden email]>, relay=dougbarton.us[private/dovecot-lmtp],
> delay=0.03, delays=0.01/0/0/0.01, dsn=5.1.1, status=bounced (host
> dougbarton.us[private/dovecot-lmtp] said: 550 5.1.1 <[hidden email]>
> User doesn't exist: [hidden email] (in reply to RCPT TO command))

So Dovecot has no idea how to deliver <[hidden email]>, if
that's the correct mailbox address, then your problem is with
Dovecot, but if upstream aliases(5) rewriting failed to happen in
Postfix, see above.

> From further reading it seems that I need to add some sort of additional
> mapping, but it's not clear to me what.

Either more virtual_alias_maps entries (I authoritatively recommend
avoiding aliases(5) for simple user->user mappings, that's best
done in virtual(5)), or once again override mailbox_transport instead
of local_transport.

> Adding my virtual_maps file to local_recipient_maps didn't work.

And is completely unnecessary, since always implicit, and the
problem was not recipient validation, see above.

> I also tried 'local_recipient_maps = '
> to see if I could rule out a chroot issue, but that didn't work either.

The local_recipient_maps parameter is used to reject non-existent
users (and thus avoid backscatter) during the SMTP dialogue in the
Postfix SMTP server, not in Dovecot.

> Here is postconf -n with security-related and boring items removed.
>
> alias_maps = hash:/etc/aliases

Check this for any relevant mappings.

> home_mailbox = Maildir/

This makes Postfix deliver mail to "local" user accounts to
$HOME/Maildir unless preempted by other settings.

> local_recipient_maps = proxy:unix:passwd.byname $alias_maps

This is not pertinent to the problem.

> mydomain = dougbarton.us
> mydestination = $mydomain, localhost.$mydomain, localhost

This is why local_transport, etc. are what you need to focus on.

> virtual_maps = hash:/etc/postfix/virtual_addresses

This is best expressed as:

    virtual_alias_maps = hash:/etc/postfix/virtual_addresses

and either:

    virtual_alias_domains =

or perhaps if you need to accept additional domains and rewrite to
dougbarton.us via virtual_alias_maps:

    virtual_alias_domains = dougbarton.net, dougb.net, supersetsolutions.com, dougbarton.email

in any case I strongly recommand separating virtual_alias_domains from
virtual_alias_maps.

> virtual_transport = lmtp:unix:private/dovecot-lmtp

As before this does nothing in the absence of virtual_mailbox_domains
and virtual_mailbox_maps (at least for recipient validation and
with virtual(8) also for mailbox locations).

> Any help will be greatly appreciated.

I hope this helps to put any ill will behind.

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lmtp

Doug Barton
In reply to this post by Doug Barton
Good news is that I solved the short term problem (delivery via lmtp now works).  More below. Lots of snipping to strip out what turned out to be the irrelevant bits.

--------------------------------------------
On Wed, 3/15/17, Viktor Dukhovni <[hidden email]> wrote:

 Subject: Re: Problems with lmtp
 To: [hidden email]
 Date: Wednesday, March 15, 2017, 10:15 PM
 
 On Thu, Mar 16, 2017 at
 04:02:58AM +0000, Doug wrote:
 
 >      $ postconf -d mail_version
 > 
 > Yes, 3.1.0, thank you.
 
 Cool.  I would expect that this likely contains backports of later
 patches, but unfortunately the Linux distros
 tend to avoid backporting upstream version
 number updates, so it is difficult to tell whether
 you have all the fixes from 3.1.0 to 3.1.4, but
 it is quite possible that you do.
 
[] Yeah, there are a lot of things I like about the way debian and its derivatives handle packaging, but this not one of them. :-/  

 > On my mail host I have 1 normal user. I have postfix configured to accept
 > mail for several different domains, and each domain has a lot of different
 > mail usernames (I use this for mailing lists and such). I use the
 > virtual_maps feature of postfix, and have a map file that looks like this:
 
 > All of this works great, and mail for all the different usernames and
 > domains gets delivered into my one real user's Maildir, and I can see the
 > mail with my IMAP clients.
 
 The important thing to understand here is the difference between the "local", "virtual alias" and "virtual mailbox" address classes, as explained in ADDRESS_CLASS_README.
 
[] Yeah, I think it's coming clear. I read through that tonight, need to read some more to digest better.  I see (or think I see) how the virtual_alias_domains and virtual_alias_maps would work to do the same thing I'm doing now.
 
     - Domains listed in virtual_alias_domains are exclusively  designated as holding only aliases to other real domains. Don't make the mistake of assuming that a domain must   be listed here in order for virtual_alias_maps to happen.

[] Ok, I'll bite .... what makes virtual_alias_maps happen?

 > So according to all the tutorials I've read my assumption is that my next
 > step is this in postfix' main.cf:
 >
 >     virtual_transport = lmtp:unix:private/dovecot-lmtp
 
 The reason this failed for you, is that your domains are "local"
 (listed in mydestination) so their delivery is controlled via
 $local_transport not $virtual_transport.
 
[] Yes, I've grasped that now, thank you for confirming.

 > I have since learned that I probably don't want virtual_transport for
 > this, but I probably do want local_transport. The problem is that if I
 > put in local_transport = lmtp:unix:private/dovecot-lmtp I get a bounce
 > every time:
 > Mar 15 18:01:20 dougbarton postfix/lmtp[11793]: 8BCD38F:
 > to=<[hidden email]>,relay=dougbarton.us[private/dovecot-lmtp],
 > delay=0.03, delays=0.01/0/0/0.01, dsn=5.1.1, status=bounced (host
 > dougbarton.us[private/dovecot-lmtp] said: 550 5.1.1 <[hidden email]>
 > User doesn't exist: [hidden email] (in reply to RCPT TO command))
 
 So Dovecot has no idea how to deliver <[hidden email]>, if
 that's the correct mailbox address, then your problem is with
 Dovecot

[] After a lot more testing tonight that was the problem. Short version is (as I understand it) that lmtp expects a full address ([hidden email]), which is what postfix is feeding it. The problem is then getting dovecot to understand what to do with that fully qualified user once it gets it. For my case, since the 'user' that postfix is mapping to is the same as the local Unix user I want it delivered to, the answer is to put this in dovecot.conf: auth_username_format=%n

That tells dovecot to only deal with the username portion, not the whole string.
 
 > Here is postconf -n with security-related and boring items removed.
 >
 > alias_maps = hash:/etc/aliases
 
 Check this for any relevant mappings.

[] Nothing exciting, just the default postmaster:root
 
 > home_mailbox = Maildir/
 
 This makes Postfix deliver mail to "local" user accounts to
 $HOME/Maildir unless preempted by other settings.
 
[] Yeah, been thinking that was going to be redundant when I get things working.

 > virtual_maps = hash:/etc/postfix/virtual_addresses
 
 This is best expressed as:
 
     virtual_alias_maps = hash:/etc/postfix/virtual_addresses
 
 and either:
 
   virtual_alias_domains =
 
 or perhaps if you need to accept additional domains and rewrite to
 dougbarton.us via virtual_alias_maps:
 
 virtual_alias_domains = dougbarton.net, dougb.net, supersetsolutions.com, dougbarton.email
 
 in any case I strongly recommand separating
 virtual_alias_domains from virtual_alias_maps.

[] Ok, I think I'm getting it now. Once I solve the lmtp problem I will tackle making this stuff more rational. It sounds like my plan is to do the following:

1. Keep all the domains in mydestination since I want them all locally delivered.
2. s/virtual_maps/virtual_alias_maps/
3. virtual_alias_domains=

Does that sound about right? I'm not brave enough to do that now since I need sleep, but I will give it a try tomorrow when I can watch the logs in the background.
 
 I hope this helps to put any ill will behind.

[] Indeed, thank you.  

Doug
Reply | Threaded
Open this post in threaded view
|

Updating Postfix was: Re: Problems with lmtp

Scott Kitterman-4
In reply to this post by Viktor Dukhovni


On March 16, 2017 1:15:54 AM EDT, Viktor Dukhovni <[hidden email]> wrote:

>On Thu, Mar 16, 2017 at 04:02:58AM +0000, Doug wrote:
>
>>      $ postconf -d mail_version
>>  
>> Yes, 3.1.0, thank you.
>
>Cool.  I would expect that this likely contains backports of later
>patches, but unfortunately the Linux distros tend to avoid backporting
>upstream version number updates, so it is difficult to tell whether
>you have all the fixes from 3.1.0 to 3.1.4, but it is quite possible
>that you do.

...

He doesn't (not that it turned out to matter this time).  There's nothing added to the package Ubuntu got from Debian that related to upstream fixes.

That said, back when I was involved in Ubuntu development, I did secure special permission to update Postfix for third digit updates based on the demonstrated history of such updates being confined to actual bug fixes and being high quality with low regression risk.

Since I stopped working on Ubuntu, no one has done the work.  I know there are more than a few Ubuntu users here.  If you want the distribution to deliver post-release updates of Postfix, then all it takes is someone volunteering to do the work.

I don't have time to do much in the way of training, but if anyone is interested in taking this up, feel free to contact me off list and I will point you in the right direction.

Scott K
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lmtp

Viktor Dukhovni
In reply to this post by Doug Barton
On Thu, Mar 16, 2017 at 08:56:20AM +0000, Doug wrote:

> >  >      $ postconf -d mail_version
> >  > 
> >  > Yes, 3.1.0, thank you.
> >  
> >  Cool.  I would expect that this likely contains backports of later
> >  patches, but unfortunately the Linux distros tend to avoid backporting
> >  upstream version number updates, so it is difficult to tell whether you
> >  have all the fixes from 3.1.0 to 3.1.4, but it is quite possible that
> >  you do.
>
> [] Yeah, there are a lot of things I like about the way debian and its
> derivatives handle packaging, but this not one of them. :-/

Turns out (per Scott Kitterman that Ubuntu may not in general
backport fixes from Postfix patch releases.  That's a shame.
Relevant post-release updates include:
       
        20160310
       
                Bugfix (introduced: Postfix 2.6): the Milter SMFIR_CHGFROM
                (replace sender) request lost the sender_bcc_maps address.
                Fixed by moving some record keeping to the sender output
                function.  Files: cleanup/cleanup_envelope.c,
                cleanup/cleanup_addr.c, cleanup/cleanup_milter.c,
                cleanup/cleanup.h, regression tests.
       
        20160410
       
                Bugfix (introduced: Postfix 2.6): the "bad filetype"
                header_checks pattern falsely rejected Content-Mumble headers
                with ``name="example"; x-apple-part-url="example.com"''.
                Fixed by respecting the ";" separator between content
                attribute values.  Reported by Cedric Knight.  File:
                proto/header_checks.
       
        20160619
       
                Bugfix (introduced: 20091121): with the introduction of
                sender_dependent_default_transport_maps, the SMTP daemon
                was not updated. This resulted in false rejects with
                sender-dependent "error" transports. Based on a fix by
                Russell Yanofsky.  Files: global/resolve_clnt.c,
                global/resolve_clnt.h, smtpd/smtpd_check.c, smtpd/smtpd_check.h,
                smtpd/smtpd_milter.c, smtpd/smtpd_resolve.c, smtpd/smtpd_resolve.h.
       
        20160717
       
                Bugfix (introduced: Postfix 1.1): the virtual(8) delivery
                agent discarded the error result from vstream_fseek().
                File: virtual/mailbox.c.
       
        20160730
       
                Bugfix (introduced: 20090614): with concurrent connections
                from the same client IP address, and after-220 tests enabled,
                postscreen could overwrite the cached "all tests completed"
                result of one connection that completed the after-220 tests,
                with the "some tests not completed" result of a concurrent
                connection where the client hung up later, without completing
                the after-220 tests.
       
        20160821
       
                Bugfix (introduced: Postfix 3.0): the tls_session_ticket_cipher
                documentation says aes-256-cbc, but the implementation was
                using aes-128-cbc (note that Postfix session ticket keys
                are rotated after 1/2 hour, to limit the impact of attacks
                on session ticket keys).
       
        20160911
       
                Bugfix (introduced: Postfix 3.0): the SMTP daemon did not
                reset a previous session's command counts before rejecting
                a client that exceeds request or concurrency rates. File:
                smtpd/smtpd.c.
       
        20160917
       
                Bugfix (introduced: Postfix 3.0): the unionmap did not
                propagate table lookup errors.  Based on patch by Roel van
                Meer.  Files: util/dict_union.c, util/dict_union_test.*.
       
        20160925
       
                Workaround (problem introduced: Postfix 2.11): to avoid
                false "not found" errors with MySQL map queries that contain
                UTF8-encoded text, specify "option_group = client" in Postfix
                MySQL configuration files.  This will be the default setting
                with Postfix 3.2 and later.
       
        20161105
       
                Bugfix (introduced: Postfix 1.1): the postsuper command did
                not count a successful rename operation after error recovery.
                Problem reported by Markus Schönhaber. File: postsuper/postsuper.c.
       
        20161204
       
                Bugfix (introduced: Postfix 3.1): cut-and-paste error in
                the "postfix tls deploy-server-cert" command, causing the
                wrong certfile and keyfile to be used. Viktor Dukhovni.
                File: conf/postfix-tls-script.
       
                Robustness: create a new keyfile when "postfix tls
                new-server-cert" is invoked and main.cf specifies a
                non-existent keyfile. Viktor Dukhovni.  File:
                conf/postfix-tls-script.
       
        20161206
       
                Bugfix (introduced: Postfix 3.0): when receiving a MAIL
                FROM...SMTPUTF8 command while smtpd_delay_reject=no, enable
                SMTPUTF8 support before processing smtpd_sender_restrictions.
                Problem reported by Viktor Dukhovni. File: smtpd/smtpd.c.
       
        20161220
       
                Bugfix (introduced: Postfix 2.1.0): the Postfix SMTP daemon
                did not query sender_canonical_maps when rejecting unknown
                senders with "smtpd_reject_unlisted_recipient = yes" or
                with reject_unlisted_sender.  Stephen R. van den Berg (Mr.
                procmail).  Files: smtpd/smtpd.c, smtpd/smtpd_check.c.

> >  The important thing to understand here is the difference between the
> >  "local", "virtual alias" and "virtual mailbox" address classes, as
> >  explained in ADDRESS_CLASS_README.
>  
> Yeah, I think it's coming clear. I read through that tonight, need to
> read some more to digest better.  I see (or think I see) how the
> virtual_alias_domains and virtual_alias_maps would work to do the same
> thing I'm doing now.

Indeed, all my domains are virtual alias domains, whose valid
recipients are rewritten to the synthetic virtual mailbox domain
"virtual.invalid", which is delivered to Dovecot.  I don't use
local(8) for delivery, but it would be useful if I had to support
mailing lists, vacation, ...  That can always be added by rewriting
addresses to a "local.invalid" domain, and delivering only that
domain locally.

    main.cf:
        indexed = ${default_database_type}:${config_directory}/
        transport_maps = ${indexed}transport

        # All real domains are virtual alias domains
        #
        virtual_alias_domains = ${indexed}vdomains
        virtual_alias_maps = ${indexed}valiases

        # No real virtual mailbox domains, but virtual mailbox
        # delivery still happens via rewrites into "virtual.invalid"
        # and the aid of a transport table entry.
        #
        virtual_mailbox_domains =

        # Mailbox storage configuration for virtual(8), not needed when
        # delivery is to Dovecot via LMTP.  See also message_size_limit
        #
        virtual_mailbox_limit = 209715200
        virtual_mailbox_maps = ${indexed}vmbox
        virtual_mailbox_base = /var/spool/virtual
        # When using the built-in virtual(8) delivery agent, these need
        # to match the uid/gid used by Dovecot to manage maildirs
        virtual_uid_maps = static:500
        virtual_gid_maps = static:500

        # No real local domains, but local(8) delivery still happens via
        # rewrites into "local.invalid" and the aid of a transport table
        # entry
        mydestination =
        local_transport = error:5.1.2 Mailbox unavailable
        local_recipient_maps =
        alias_database = ${indexed}aliases
        alias_maps = $alias_database

    transport:
        # Postfix built-in virtual(8) delivery
        virtual.invalid virtual
        local.invalid local

    vdomains:
        example.org virtual alias domain
        example.net virtual alias domain
        ...

    valiases:
        [hidden email] [hidden email]
        [hidden email] [hidden email]
        [hidden email] [hidden email]
        # virtual alias mapping is recursive
        [hidden email] [hidden email], [hidden email]
        [hidden email] [hidden email]
        [hidden email] [hidden email]

    vmbox:
        [hidden email] user1/
        [hidden email] user2/

    aliases:
        owner-somelist: [hidden email]
        somelist: :include:/some/user/listfile


>     - Domains listed in virtual_alias_domains are exclusively 
> designated as holding only aliases to other real domains. Don't
> make the mistake of assuming that a domain must   be listed here
> in order for virtual_alias_maps to happen.
>
> [] Ok, I'll bite .... what makes virtual_alias_maps happen?

The use of virtual alias maps happens for all recipients, as part
of mail entering the queue via cleanup(8).  Not dependent on the
address class.  It is possible to disable this (with care) via
receive_overrride_options, either before or after a content_filter.
See FILTER_README for details.

> >  So Dovecot has no idea how to deliver <[hidden email]>, if
> >  that's the correct mailbox address, then your problem is with
> >  Dovecot
>
> After a lot more testing tonight that was the problem. Short version
> is (as I understand it) that lmtp expects a full address ([hidden email]),
> which is what postfix is feeding it.

Yes.

> The problem is then getting dovecot to understand what to do with that
> fully qualified user once it gets it.  For my case, since the 'user' that
> postfix is mapping to is the same as the local Unix user I want it delivered
> to, the answer is to put this in dovecot.conf: auth_username_format=%n

Or similar, yes.  I have:

    userdb {
        args = uid=500 gid=500 home=/var/spool/virtual mail=maildir:/var/spool/virtual/%n
        driver = static
    }

> Ok, I think I'm getting it now. Once I solve the lmtp problem I will
> tackle making this stuff more rational. It sounds like my plan is to do
> the following:
>
> 1. Keep all the domains in mydestination since I want them all locally delivered.

Not required, see above.  Postfix has a notion of "final" domains,
which subsumes "virtual alias", "virtual mailbox" and "local"
domains.  You can use any combination of these for "locally delivered"
mail.  I tend to keep mydestination empty.   See also the null-client
walk-throgh in MULTI_INSTANCE_README.

> 2. s/virtual_maps/virtual_alias_maps/
> 3. virtual_alias_domains=

Yes, or, if you prefer, make *all* the "real" domains virtual alias,
and use synthetic domains for delivery.  See above.

One disadvantage (or advantage if you like that) of mydestination,
is that the namespace of users (presumed local unix accounts) is
the same in all domains listed in mydestination.  So if you have
example.com and example.net both in mydestination, and user "bob"
has a "[hidden email]" email address, then necessarily "[hidden email]"
is also bob's address.

With virtual alias or virtual mailbox domains each domain's namespace
is independent.

> Does that sound about right? I'm not brave enough to do that now since I
> need sleep, but I will give it a try tomorrow when I can watch the logs
> in the background.

Sounds reasonable to me.

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lmtp

Doug Barton
In reply to this post by Doug Barton

--------------------------------------------
On Thu, 3/16/17, Viktor Dukhovni <[hidden email]> wrote:

 Subject: Re: Problems with lmtp
 To: [hidden email]
 Date: Thursday, March 16, 2017, 8:08 AM
 
 On Thu, Mar 16, 2017 at
 08:56:20AM +0000, Doug wrote:
 
 > >  The important thing to understand here is the difference between the
 > >  "local", "virtual alias" and "virtual mailbox" address classes, as
 > >  explained in ADDRESS_CLASS_README.
 > 
 > Yeah, I think it's coming clear. I read through that tonight, need to
 > read some more to digest better.  I see (or think I see) how the
 > virtual_alias_domains and virtual_alias_maps would work to do the same
 > thing I'm doing now.
 
 Indeed, all my domains are virtual alias domains, whose valid
 recipients are rewritten to the synthetic virtual mailbox domain
 "virtual.invalid", which is delivered to Dovecot.    I don't use
local(8) for delivery, but it would be useful if I had to support
mailing lists, vacation, ...  That can always be added by rewriting
addresses to a "local.invalid" domain, and delivering only that
domain locally.

[] I am very interested in your example, and it looks like the way I want to go, even though I don't need virtual mailboxes. Thank you for sharing it. FWIW, Sieve has a notion of 'Vacation' that looks adequate, although I didn't dig into it much as it's not an issue for me. I'm impressed with the filtering capabilities so far though, and very glad to finally move that to server-side. :)
 
 >     - Domains listed in virtual_alias_domains are exclusively 
 >     designated as holding only aliases to other real domains. Don't
 >     make the mistake of assuming that a domain must   be listed here
 >     in order for virtual_alias_maps to happen.
 >
 > [] Ok, I'll bite .... what makes virtual_alias_maps happen?
 
 The use of virtual alias maps happens for all recipients, as part
 of mail entering the queue via cleanup(8).  Not dependent on the
 address class. 

[] Ah, Ok, thanks. One less thing to be concerned about then.
 
 > Ok, I think I'm getting it now. Once I solve the lmtp problem I will
 > tackle making this stuff more rational. It sounds like my plan is to do
 > the following:
 >
 > 1. Keep all the domains in mydestination since I want them all locally delivered.
 
 Not required, see above.  Postfix has a notion of "final" domains,
 which subsumes "virtual alias", "virtual mailbox" and "local" domains.  You can use any
 combination of these for "locally delivered" mail.  I tend to keep mydestination
 empty.   See also the null-client walk-throgh in MULTI_INSTANCE_README.
 
[] Ok, thanks.

 > 2. s/virtual_maps/virtual_alias_maps/
 > 3. virtual_alias_domains=
 
 Yes, or, if you prefer, make *all* the "real" domains virtual alias,
 and use synthetic domains for delivery.  See above.
 
[] Yeah, more homework to do. Thanks again.

Doug
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lmtp

chaouche yacine
In reply to this post by Viktor Dukhovni
On Thursday, March 16, 2017 4:09 PM, Viktor Dukhovni <[hidden email]> wrote:

>> The problem is then getting dovecot to understand what to do with that
>> fully qualified user once it gets it.  For my case, since the 'user' that
>> postfix is mapping to is the same as the local Unix user I want it delivered
>> to, the answer is to put this in dovecot.conf: auth_username_format=%n
>
> Or similar, yes.  I have:
>
>    userdb {
>    args = uid=500 gid=500 home=/var/spool/virtual mail=maildir:/var/spool/virtual/%n
>    driver = static
>    }


Sorry for asking this on a postfix list but Viktor it seems all your users share the same home directory ? what about sieve scripts ?

-- Yassine.
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lmtp

Thomas Leuxner
* chaouche yacine <[hidden email]> 2017.03.17 10:25:

> > Or similar, yes.  I have:
> >
> >    userdb {
> >    args = uid=500 gid=500 home=/var/spool/virtual mail=maildir:/var/spool/virtual/%n
> >    driver = static
> >    }
>
>
> Sorry for asking this on a postfix list but Viktor it seems all your users share the same home directory ? what about sieve scripts ?
>
> -- Yassine.
The example used by Viktor does not invoke Dovecot's LDA/LMTP as MDA. Postfix performs final delivery, thus Sieve scripts can't be used.

 vmbox:
        [hidden email] user1/
        [hidden email] user2/

Regards
Thomas

signature.asc (188 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lmtp

chaouche yacine
Thank you Thomas, so if I understand correctly in Viktor's config dovecot is only used by postfix as a backend to query for valid virtual email addresses ?
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lmtp

Viktor Dukhovni

> On Mar 17, 2017, at 9:52 AM, chaouche yacine <[hidden email]> wrote:
>
> Thank you Thomas, so if I understand correctly in Viktor's config dovecot is only used by postfix as a backend to query for valid virtual email addresses ?

Dovecot is used as an IMAP server.  Postfix does maildir delivery.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: Problems with lmtp

Thomas Leuxner
In reply to this post by chaouche yacine
* chaouche yacine <[hidden email]> 2017.03.17 14:52:

> Thank you Thomas, so if I understand correctly in Viktor's config dovecot is only used by postfix as a backend to query for valid virtual email addresses ?

Hi Yassine,

one of the benefits of using Dovecot's MDAs besides Sieve, is that they update the metadata and indexes upon delivery which improves its IMAP performance. You also get a broader choice of mailstorage formats which offer new features compared to the older maildir format.

Regards
Thomas

signature.asc (188 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Problems with lmtp

chaouche yacine
All clear ! thanks.


On Friday, March 17, 2017 5:19 PM, Thomas Leuxner <[hidden email]> wrote:


* chaouche yacine <[hidden email]> 2017.03.17 14:52:

> Thank you Thomas, so if I understand correctly in Viktor's config dovecot is only used by postfix as a backend to query for valid virtual email addresses ?

Hi Yassine,

one of the benefits of using Dovecot's MDAs besides Sieve, is that they update the metadata and indexes upon delivery which improves its IMAP performance. You also get a broader choice of mailstorage formats which offer new features compared to the older maildir format.

Regards

Thomas