Keep Postfix running in the foreground

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Keep Postfix running in the foreground

A Debian User

Hello,

I am currently having trouble to get postfix running in a Docker Container.

Docker requires a Process to stay alive and in foreground at ID 1, if not the container dies.

Is there any way to make it stay in the foreground, like it is possible for instance with apachectl -DFOREGROUND (Without scripts, supervisor, ...)?

Many thanks for helpful replies in advance!




Reply | Threaded
Open this post in threaded view
|

Re: Keep Postfix running in the foreground

Wietse Venema
A Debian User:

> Hello,
>
> I am currently having trouble to get postfix running in a Docker Container.
>
> Docker requires a Process to stay alive and in foreground at ID 1, if
> not the container dies.
>
> Is there any way to make it stay in the foreground, like it is possible
> for instance with _/apachectl -DFOREGROUND/_ (Without scripts,
> supervisor, ...)?
>
> Many thanks for helpful replies in advance!

That is not currently supported. You are welcome to send a patch
for the postfix-script file, for example to implement a "run"
command.  I don't see, however, how one would use this to implement
other management commands (postfix reload, postconf -e, postsuper,
etc.).

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Keep Postfix running in the foreground

Viktor Dukhovni
In reply to this post by A Debian User

> On Sep 26, 2017, at 3:21 PM, A Debian User <[hidden email]> wrote:
>
> Hello,
>
> I am currently having trouble to get postfix running in a Docker Container.
>
> Docker requires a Process to stay alive and in foreground at ID 1, if not the container dies.
>
> Is there any way to make it stay in the foreground, like it is possible for instance with apachectl -DFOREGROUND (Without scripts, supervisor, ...)?
>
> Many thanks for helpful replies in advance!

This is not supported or easy.  In multi-instance deployments Postfix
runs multiple independent master(8) processes.  None of them would be
a natural distinguished foreground top-level process.  There's no
"meta-master" that sticks around to manage all the underlying masters.

Even in a single-instance deployment, I don't see a suitable master(8)
option for staying in the foreground.  You might just need a dummy
process that launches Postfix as a background child process, and sticks
around forever, perhaps periodically trying to lock  the master lock
file and exiting if the lock is available.  This assumes that Docker
permits running background processes.  If not, then Postfix is not
a docker-compatible application.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: Keep Postfix running in the foreground

dennisthetiger
In reply to this post by A Debian User
On Tue, 26 Sep 2017 21:21:56 +0200
A Debian User <[hidden email]> wrote:

> Hello,
>
> I am currently having trouble to get postfix running in a Docker
> Container.
>
> Docker requires a Process to stay alive and in foreground at ID 1, if
> not the container dies.

I don't know much about Docker, but would it be possible to just start
a shell script therein that pretty much does nothing, e.g. 'while true
()' as PID 1?

-Dennis Carr
Reply | Threaded
Open this post in threaded view
|

Re: Keep Postfix running in the foreground

Wietse Venema
Dennis Carr:

> On Tue, 26 Sep 2017 21:21:56 +0200
> A Debian User <[hidden email]> wrote:
>
> > Hello,
> >
> > I am currently having trouble to get postfix running in a Docker
> > Container.
> >
> > Docker requires a Process to stay alive and in foreground at ID 1, if
> > not the container dies.
>
> I don't know much about Docker, but would it be possible to just start
> a shell script therein that pretty much does nothing, e.g. 'while true
> ()' as PID 1?

The Docker approach complicates Postfix multi-instance support so
we may have to forego that. What remains is to determine that Docker
shutdown, i.e. yanking the container from under a running Postfix
system, will be no worse than an ordinary system crash, i.e. it
should not result in loss of email.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Keep Postfix running in the foreground

Viktor Dukhovni


> On Dec 18, 2017, at 9:09 PM, Wietse Venema <[hidden email]> wrote:
>
> The Docker approach complicates Postfix multi-instance support so
> we may have to forego that. What remains is to determine that Docker
> shutdown, i.e. yanking the container from under a running Postfix
> system, will be no worse than an ordinary system crash, i.e. it
> should not result in loss of email.

It should not be that hard to construct a minder process for multiple
Postfix instances.  Such a thing might be useful even for single-instance
deployments to allow "postfix-restart" to work sensibly.  I don't think
such a beast should at present be bundled with Postfix.  This can be
a separate project.  All that Postfix needs to provide is a foreground
master(8) daemon.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: Keep Postfix running in the foreground

Wietse Venema
Viktor Dukhovni:

>
>
> > On Dec 18, 2017, at 9:09 PM, Wietse Venema <[hidden email]> wrote:
> >
> > The Docker approach complicates Postfix multi-instance support so
> > we may have to forego that. What remains is to determine that Docker
> > shutdown, i.e. yanking the container from under a running Postfix
> > system, will be no worse than an ordinary system crash, i.e. it
> > should not result in loss of email.
>
> It should not be that hard to construct a minder process for multiple
> Postfix instances.  Such a thing might be useful even for single-instance
> deployments to allow "postfix-restart" to work sensibly.  I don't think
> such a beast should at present be bundled with Postfix.  This can be
> a separate project.  All that Postfix needs to provide is a foreground
> master(8) daemon.

I think that Docker fundamentally wants one service instance per
container. On Postfix service instance translates into one queue,
for example submission+smtp sharing one queue, similar to http+https
sharing one website. Let's not fight the Docker approach, and leave
orchestration to a different layer in the stack.

I'm more concerned about queue persistence. If instances are scaled
horizontally, then scaling down should not result in disappearance
of a message that was queued in a decommissioned service instance.

As for forgrounding, this must happen only after the 'postfix
check' sanity checks and repairs complete sucessfully. Running a
'bare' master daemon would violate design assumptions. So this
will require a new 'postfix' subcommand that starts exactly one
instance in the foreground.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Keep Postfix running in the foreground

Stephen Satchell
On 12/19/2017 05:25 AM, Wietse Venema wrote:
> As for forgrounding, this must happen only after the 'postfix
> check' sanity checks and repairs complete sucessfully. Running a
> 'bare' master daemon would violate design assumptions. So this
> will require a new 'postfix' subcommand that starts exactly one
> instance in the foreground.

There are a lot of daemons out there that follow an interesting rule:
If you want to stay in the foreground, there is a switch in the launch
command to stay in the foreground.  So that might be an option instead
of having yet another command in the config files.  (Or perhaps in
addition to.)

Don't you have such a thing you use to debug Postfix, Mr. Venema?
Reply | Threaded
Open this post in threaded view
|

PATCH: Keep Postfix running in the foreground

Wietse Venema
In reply to this post by Wietse Venema
Wietse Venema:
> I think that Docker fundamentally wants one service instance per
> container. On Postfix service instance translates into one queue,
> for example submission+smtp sharing one queue, similar to http+https
> sharing one website. Let's not fight the Docker approach, and leave
> orchestration to a different layer in the stack.
...
> As for forgrounding, this must happen only after the 'postfix
> check' sanity checks and repairs complete sucessfully. Running a
> 'bare' master daemon would violate design assumptions. So this
> will require a new 'postfix' subcommand that starts exactly one
> instance in the foreground.

Attached is a proof-of-concept implementation.
Manpage fragment:

POSTFIX(1)                                                          POSTFIX(1)

NAME
       postfix - Postfix control program
        ...
DESCRIPTION
        ...
       The following commands are implemented:

       check  Warn about bad directory/file ownership or permissions, and cre-
              ate missing directories.

       start  Start  the Postfix mail system. This also runs the configuration
              check described above.

       start-fg
              Like start, but keep the master  daemon  running  in  the  fore-
              ground.  This  requires  that multi-instance support is disabled
              (i.e. the multi_instance_directories parameter value is  empty).
        ...

I suppose one approach is to make a Postfix container disposable,
i.e. a container is never updated with a new Postfix version, but
it is replaced with a newer one, and it imports its queue and data
directories from the host. These directories must of course not be
imported into multiple containers. I don't know how to prevent that.

Also, a Postfix container would import the logging sockets from the
host (www.projectatomic.io/blog/2016/10/playing-with-docker-logging)
and would set 'syslog_name = $myhostname/postfix' in the container's
main.cf file to make logging from different containers distinct.
Of course the logging sockets may be imported into as many containers
as needed.

If one wants multiple Postfix instances in a single container, then
that will require a 'minder' program that runs in the foreground and
that plays nice with higher-level orchestration systems. I won't
sabotage that approach.

        Wietse

diff --exclude=man --exclude=html --exclude=README_FILES --exclude=INSTALL --exclude=.indent.pro --exclude=Makefile.in -r -ur /var/tmp/postfix-3.3-20171218/conf/postfix-script ./conf/postfix-script
--- /var/tmp/postfix-3.3-20171218/conf/postfix-script 2017-09-12 14:16:54.000000000 -0400
+++ ./conf/postfix-script 2017-12-19 09:37:18.000000000 -0500
@@ -117,7 +117,7 @@
  echo "Stop postfix"
  ;;
 
-start)
+start|start-fg)
 
  $daemon_directory/master -t 2>/dev/null || {
  $FATAL the Postfix mail system is already running
@@ -135,11 +135,28 @@
  $daemon_directory/postfix-script check-warn
  fi
  $INFO starting the Postfix mail system
- # NOTE: wait in foreground process to get the initialization status.
- $daemon_directory/master -w || {
-    $FATAL "mail system startup failed"
-    exit 1
- }
+ case $1 in
+ start)
+    # NOTE: wait in foreground process to get the initialization status.
+    $daemon_directory/master -w || {
+ $FATAL "mail system startup failed"
+ exit 1
+    }
+    ;;
+ start-fg)
+    # Foreground start-up is incompatible with multi-instance mode.
+    # We can't use "exec $daemon_directory/master" here: that would
+    # break process group management, and "postfix stop" would kill
+    # too many processes.
+    case $instances in
+    "") $daemon_directory/master
+ ;;
+     *) $FATAL "start-fg does not support multi_instance_directories"
+ exit 1
+ ;;
+    esac
+    ;;
+ esac
  ;;
 
 drain)
diff --exclude=man --exclude=html --exclude=README_FILES --exclude=INSTALL --exclude=.indent.pro --exclude=Makefile.in -r -ur /var/tmp/postfix-3.3-20171218/src/postfix/postfix.c ./src/postfix/postfix.c
--- /var/tmp/postfix-3.3-20171218/src/postfix/postfix.c 2016-09-17 10:50:56.000000000 -0400
+++ ./src/postfix/postfix.c 2017-12-19 09:19:59.000000000 -0500
@@ -31,6 +31,11 @@
 /* .IP \fBstart\fR
 /* Start the Postfix mail system. This also runs the configuration
 /* check described above.
+/* .IP \fBstart-fg\fR
+/* Like \fBstart\fR, but keep the master daemon running in the
+/* foreground. This requires that multi-instance support is
+/* disabled (i.e. the multi_instance_directories parameter
+/* value is empty).
 /* .IP \fBstop\fR
 /* Stop the Postfix mail system in an orderly fashion. If
 /* possible, running processes are allowed to terminate at
Reply | Threaded
Open this post in threaded view
|

Re: Keep Postfix running in the foreground

Wietse Venema
In reply to this post by Stephen Satchell
Stephen Satchell:

> On 12/19/2017 05:25 AM, Wietse Venema wrote:
> > As for forgrounding, this must happen only after the 'postfix
> > check' sanity checks and repairs complete sucessfully. Running a
> > 'bare' master daemon would violate design assumptions. So this
> > will require a new 'postfix' subcommand that starts exactly one
> > instance in the foreground.
>
> There are a lot of daemons out there that follow an interesting rule:
> If you want to stay in the foreground, there is a switch in the launch
> command to stay in the foreground.  So that might be an option instead
> of having yet another command in the config files.  (Or perhaps in
> addition to.)
>
> Don't you have such a thing you use to debug Postfix, Mr. Venema?

This may come as a surprise: Postfix is not just one daemon, it is
actually a collection of daemons, so the question may seem simple,
but the answer is not.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: PATCH: Keep Postfix running in the foreground

Eray Aslan-2
In reply to this post by Wietse Venema
On Tue, Dec 19, 2017 at 10:01:53AM -0500, Wietse Venema wrote:
> I suppose one approach is to make a Postfix container disposable,
> i.e. a container is never updated with a new Postfix version, but
> it is replaced with a newer one

That is the common Docker approach.  Images are immutable.

> and it imports its queue and data directories from the host. These
> directories must of course not be imported into multiple containers. I
> don't know how to prevent that.

That is a problem for a different layer of the stack.  The sysadmin is
supposed to provide persistent storage and make sure that multiple
containers do not write to the same directory.  It should not be our job
to babysit the infra.  Inform and bail out if the deal is broken?

> Also, a Postfix container would import the logging sockets from the
> host (www.projectatomic.io/blog/2016/10/playing-with-docker-logging)
> and would set 'syslog_name = $myhostname/postfix' in the container's
> main.cf file to make logging from different containers distinct.
> Of course the logging sockets may be imported into as many containers
> as needed.

Uhm, systemd (or any other init system) as pid 1 is not the "docker
way".  It is better for docker to know when the service has stopped /
crashed etc so it can take appropriate action.  So consider ditching
seperate pid 1 daemon option.

Usually sending log output to console is the preferred approach.  If we
cannot do it natively, having syslog daemon write to console (in
addition to local log file?) looks like a better option than importing
sockets from the host.

> If one wants multiple Postfix instances in a single container, then
> that will require a 'minder' program that runs in the foreground and
> that plays nice with higher-level orchestration systems. I won't
> sabotage that approach.

Do we really need that?  Too many layers all trying to do a similar job?

--
Eray
Reply | Threaded
Open this post in threaded view
|

Re: PATCH: Keep Postfix running in the foreground

Wietse Venema
Eray Aslan:

> On Tue, Dec 19, 2017 at 10:01:53AM -0500, Wietse Venema wrote:
> > I suppose one approach is to make a Postfix container disposable,
> > i.e. a container is never updated with a new Postfix version, but
> > it is replaced with a newer one
>
> That is the common Docker approach.  Images are immutable.
>
> > and it imports its queue and data directories from the host. These
> > directories must of course not be imported into multiple containers. I
> > don't know how to prevent that.
>
> That is a problem for a different layer of the stack.  The sysadmin is
> supposed to provide persistent storage and make sure that multiple
> containers do not write to the same directory.  It should not be our job
> to babysit the infra.  Inform and bail out if the deal is broken?

Postfix will bail out if it knows that the queue or data directory
are shared, because that can result in data corruption.

How do I enforce that constraint when directories are imported into
a container from the host?

> > Also, a Postfix container would import the logging sockets from the
> > host (www.projectatomic.io/blog/2016/10/playing-with-docker-logging)
> > and would set 'syslog_name = $myhostname/postfix' in the container's
> > main.cf file to make logging from different containers distinct.
> > Of course the logging sockets may be imported into as many containers
> > as needed.
>
> Uhm, systemd (or any other init system) as pid 1 is not the "docker
> way".  It is better for docker to know when the service has stopped /
> crashed etc so it can take appropriate action.  So consider ditching
> seperate pid 1 daemon option.

THIS ARTICLE IS NOT ABOUT RUNNING SYSTEMD IN THE CONTAINER.
Or any other non-Postfix daemon, for that matter.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: PATCH: Keep Postfix running in the foreground

Viktor Dukhovni
In reply to this post by Eray Aslan-2


> On Dec 19, 2017, at 10:19 AM, Eray Aslan <[hidden email]> wrote:
>
> Usually sending log output to console is the preferred approach.  If we
> cannot do it natively, having syslog daemon write to console (in
> addition to local log file?) looks like a better option than importing
> sockets from the host.

Not an option for Postfix.  Multiple processes generate log messages
concurrently, a shared output stream would result in interleaving of
message fragments.  The syslogd service receives discrete messages
and serializes output.  It is important to use a datagram syslog
service, to preserve chronological message order.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: PATCH: Keep Postfix running in the foreground

Wietse Venema
Viktor Dukhovni:

>
>
> > On Dec 19, 2017, at 10:19 AM, Eray Aslan <[hidden email]> wrote:
> >
> > Usually sending log output to console is the preferred approach.  If we
> > cannot do it natively, having syslog daemon write to console (in
> > addition to local log file?) looks like a better option than importing
> > sockets from the host.
>
> Not an option for Postfix.  Multiple processes generate log messages
> concurrently, a shared output stream would result in interleaving of
> message fragments.  The syslogd service receives discrete messages
> and serializes output.  It is important to use a datagram syslog
> service, to preserve chronological message order.

It should be possible to run an in-container process that receives
syslog datagrams from all Postfix daemons, and that sends complete
text records one-by-one, as long as there is only one thread that
writes to stdout/stderr.

However, I have concerns about the performance cost of multiple
layers of forwarding that handles the logging from hundreds of
concurrent Postfix daemon processes, on top of a Linux logging
infrastructure that is already insanely expensive.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: PATCH: Keep Postfix running in the foreground

Eray Aslan-2
In reply to this post by Wietse Venema
On Tue, Dec 19, 2017 at 10:53:38AM -0500, Wietse Venema wrote:
> Postfix will bail out if it knows that the queue or data directory
> are shared, because that can result in data corruption.
>
> How do I enforce that constraint when directories are imported into
> a container from the host?

I don't think you can.  But how is that different from the current
status quo?  We can only continue doing what we have been doing so far.

For the record, while running postfix in a container is certainly doable
and should probably be given as an option, I am not convinced that it is
such a good idea for any serious MTA work.

--
Eray

Reply | Threaded
Open this post in threaded view
|

Re: PATCH: Keep Postfix running in the foreground

Wietse Venema
Eray Aslan:
> On Tue, Dec 19, 2017 at 10:53:38AM -0500, Wietse Venema wrote:
> > Postfix will bail out if it knows that the queue or data directory
> > are shared, because that can result in data corruption.
> >
> > How do I enforce that constraint when directories are imported into
> > a container from the host?
>
> I don't think you can.  But how is that different from the current
> status quo?  We can only continue doing what we have been doing so far.

The status quo (prior to containers) is that Postfix prevents
unnecessary requests for support from people who do things like
sharing a queue_directory or data_directory.

The Postfix master relies on opening a 'lock' file below the queue
and data directories, and holding an exclusive lock until the process
terminates. If that works across container boundaries, then that
is one less thing to worry about. But all this is speculation until
it is proven to work.

        Wietse