Finding lost e-mails collected during misconfiguration.

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Finding lost e-mails collected during misconfiguration.

Timothy D. Legg
Hello,

Glad to be back here again.  I migrated my server to a Raspberry PI and
away from my aging EeePC netbook that was running Ubuntu Server.  I am
virtual hosting email across a few domains I own using MariaDB to store
the virtual aliases  and virtual user's information.  Ultimately, e-mail
is read within Squirrelmail.

I have to admit, it took a risky path towards configuring (I blindly
copied the configuration files for Dovecot/Postfix from the old machine).
Both packages on Raspbian were a downgrade to a lower version number.  It
didn't work.  I was due to fly to another country and be offline for
several days and I had run out of time.  Since this only affected my
email, I decided to table the project during this time.

Upon returning, I found that the problem was that I failed to give the new
SQL password to Dovecot, so it was impossible for me to authenticate via
IMAP.  Upon fixing that, I was able to log in and see new, very fresh,
e-mails, but none of the messages from June 9 through June 23 are present.

To further complicate things, I'm using maildir to store the data which
isn't nearly as user readable as mbox.  So I'm not certain where to look
to find staged e-mail that didn't end up where it was supposed to.  Are
these e-mails still located on my machine somewhere or did they get sent
down a /dev/null somewhere?

In conclusion, this loss of e-mail was an educating experience because
since the misconfiguration was with Dovecot, I would have expected the
mail to be there waiting for me once I corrected Dovecot and established
an IMAP connection.  So in the end, I realize that I underestimated how
intimately Postfix and Dovecot interact with each other.

The upside of all this, I am documenting this on an open-wiki on my
always-ad-free private site that I can share for peer review in a few days
once I brush up on the Postfix/Dovecot end of the HOWTO document.

Thanks,

Timothy D Legg


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Finding lost e-mails collected during misconfiguration.

Wietse Venema
Timothy D Legg:
[ Charset ISO-8859-1 converted... ]

> Hello,
>
> Glad to be back here again.  I migrated my server to a Raspberry PI and
> away from my aging EeePC netbook that was running Ubuntu Server.  I am
> virtual hosting email across a few domains I own using MariaDB to store
> the virtual aliases  and virtual user's information.  Ultimately, e-mail
> is read within Squirrelmail.
>
> I have to admit, it took a risky path towards configuring (I blindly
> copied the configuration files for Dovecot/Postfix from the old machine).
> Both packages on Raspbian were a downgrade to a lower version number.  It
> didn't work.  I was due to fly to another country and be offline for
> several days and I had run out of time.  Since this only affected my
> email, I decided to table the project during this time.
>
> Upon returning, I found that the problem was that I failed to give the new
> SQL password to Dovecot, so it was impossible for me to authenticate via
> IMAP.  Upon fixing that, I was able to log in and see new, very fresh,
> e-mails, but none of the messages from June 9 through June 23 are present.
>
> To further complicate things, I'm using maildir to store the data which
> isn't nearly as user readable as mbox.  So I'm not certain where to look
> to find staged e-mail that didn't end up where it was supposed to.  Are
> these e-mails still located on my machine somewhere or did they get sent
> down a /dev/null somewhere?

Postfix logs all mail deliveries, successful or not. That will tell
you whether mail was handed off to Dovecot, returned to sender, or
otherwise. Dovecot should produce logs as well.

Absent logs, you may want to run a tool that lists all files with
modification time stamps in the expected time range.

find /some/where -newer file1 -not -newer file2

With file1 and file2 created by suitable instances of "touch -t
YYMMDDhhmm".

        Wietse

> In conclusion, this loss of e-mail was an educating experience because
> since the misconfiguration was with Dovecot, I would have expected the
> mail to be there waiting for me once I corrected Dovecot and established
> an IMAP connection.  So in the end, I realize that I underestimated how
> intimately Postfix and Dovecot interact with each other.
>
> The upside of all this, I am documenting this on an open-wiki on my
> always-ad-free private site that I can share for peer review in a few days
> once I brush up on the Postfix/Dovecot end of the HOWTO document.
>
> Thanks,
>
> Timothy D Legg
>
>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Finding lost e-mails collected during misconfiguration.

Bill Cole-3
In reply to this post by Timothy D. Legg
On 26 Jun 2017, at 8:03, Timothy D Legg wrote:

> Hello,
>
> Glad to be back here again.  I migrated my server to a Raspberry PI
> and
> away from my aging EeePC netbook that was running Ubuntu Server.  I am
> virtual hosting email across a few domains I own using MariaDB to
> store
> the virtual aliases  and virtual user's information.  Ultimately,
> e-mail
> is read within Squirrelmail.

If Dr. Venema's suggestion doesn't help, please see
http://www.postfix.org/DEBUG_README.html#mail

It is impossible for any of us to know what your config is and what is
in your logs unless you tell us.

> I have to admit, it took a risky path towards configuring (I blindly
> copied the configuration files for Dovecot/Postfix from the old
> machine).
> Both packages on Raspbian were a downgrade to a lower version number.  
> It
> didn't work.  I was due to fly to another country and be offline for
> several days and I had run out of time.  Since this only affected my
> email, I decided to table the project during this time.
>
> Upon returning, I found that the problem was that I failed to give the
> new
> SQL password to Dovecot, so it was impossible for me to authenticate
> via
> IMAP.  Upon fixing that, I was able to log in and see new, very fresh,
> e-mails, but none of the messages from June 9 through June 23 are
> present.

And most of those are very likely to be lost.

If you had Postfix itself delivering to the Maildirs, Dovecot's lack of
access to user mapping would not prevent delivery. That you don't see
those messages implies that you're using Dovecot to do delivery, either
calling its LDA directly from Postfix or via LMTP. In that circumstance,
delivery failure would normally cause Postfix to requeue the message and
retry delivery later, up to a configurable period of time. When that has
expired, Postfix will bounce the message to the sender.

> To further complicate things, I'm using maildir to store the data
> which
> isn't nearly as user readable as mbox.

That's quite subjective, as Maildir has one file per message and one
directory per mailbox, with the names of both expressing useful
metadata.

> So I'm not certain where to look
> to find staged e-mail that didn't end up where it was supposed to.  
> Are
> these e-mails still located on my machine somewhere or did they get
> sent
> down a /dev/null somewhere?

That's config-dependent but if they are anywhere, they would most likely
be in Postfix's 'deferred' queue. You can check using the postqueue
command and force an immediate delivery attempt for anything still being
deferred.

> In conclusion, this loss of e-mail was an educating experience because
> since the misconfiguration was with Dovecot, I would have expected the
> mail to be there waiting for me once I corrected Dovecot and
> established
> an IMAP connection.  So in the end, I realize that I underestimated
> how
> intimately Postfix and Dovecot interact with each other.

How intimate that interaction is actually varies a great deal based on
config. For example, my personal system uses both but Postfix only
relies on Dovecot for submission auth. Postfix itself (or for some
users, procmail) delivers to Maildir trees for users whether Dovecot is
running well, running broken, or dead. Since that system has only a
handful of IMAP users, I have no need for the added complexity and
overhead of a RDBMS. It's not clear to me whether you might have a
similarly tiny number of users but if so you may find it useful to
simplify the system, particularly because you are on a constricted
physical platform.

Loading...