postfix queue perms' control for pflogsumm reporting? avoiding "warning: mail_queue_enter: create file maildrop/...: Permission denied"

[PGNet Dev]

on a new, from-distro-pkgs install of Postfix, i've noted an intermittent perms problem

it'll run just fine for quite awhile, then I start seeing a steady stream of

        ...
        Oct 28 15:02:40 svr019 postfix/postdrop[64624]: warning: mail_queue_enter: create file maildrop/553726.64624: Permission denied
        Oct 28 15:02:45 svr019 postfix/postdrop[32688]: warning: mail_queue_enter: create file maildrop/766615.32688: Permission denied
        ...

I don't intentionally/explicitly 'do' anything with maildrop/postdrop, and am not yet sure what triggers the problem.

spool/queue perms are an obvious 1st suspect ...

as installed, atm, i've

        ls -al /var/spool/postfix/
                total 64K
                drwxr-xr-x 16 root    root     4.0K Aug 31 02:52 ./
                drwxr-xr-x 14 root    root     4.0K Oct 25 19:05 ../
                drwx------  2 postfix root     4.0K Oct 28 07:51 active/
                drwx------  2 postfix root     4.0K Oct 26 16:43 bounce/
                drwx------  2 postfix root     4.0K Aug 31 02:52 corrupt/
                drwx------ 18 postfix root     4.0K Sep  1 15:35 defer/
                drwx------ 18 postfix root     4.0K Sep  1 15:35 deferred/
                drwx------  2 postfix root     4.0K Aug 31 02:52 flush/
                drwx------  2 postfix root     4.0K Aug 31 02:52 hold/
                drwx------  2 postfix root     4.0K Oct 28 08:00 incoming/
                drwx-wx---  2 postfix postdrop 4.0K Oct 27 17:00 maildrop/
                drwxr-xr-x  2 root    root     4.0K Oct 28 08:04 pid/
                drwx------  2 postfix root     4.0K Oct 28 08:04 private/
                drwx--x---  2 postfix postdrop 4.0K Oct 28 08:04 public/
                drwx------  2 postfix root     4.0K Aug 31 02:52 saved/
                drwx------  2 postfix root     4.0K Oct 17 04:54 trace/

        ls -al `which postfix` `which postqueue`
                -rwxr-xr-x 1 root root     21K Aug 31 02:52 /usr/sbin/postfix*
                -rwxr-sr-x 1 root postdrop 25K Aug 31 02:52 /usr/sbin/postqueue*

I'm mostly sure (still checking) that my 'culprit' is an overly aggressive 'pflogsumm.service'; changing,

        [Unit]
        Description=Daily pflogsumm report

        [Service]
        Type=oneshot
        ExecStart=/usr/local/etc/postfix/scripts/pflogsumm-report.sh

        KillMode=process

        LockPersonality=yes
        MemoryDenyWriteExecute=yes
- NoNewPrivileges=yes
+ NoNewPrivileges=no
        PrivateDevices=yes
        PrivateTmp=yes
        ProtectControlGroups=yes
        ProtectHome=yes
        ProtectKernelLogs=yes
        ProtectKernelModules=yes
        ProtectKernelTunables=yes
        ProtectSystem=strict
        ReadWritePaths=/var/spool/postfix/
        RestrictNamespaces=yes
        RestrictRealtime=yes

seems to do the trick; at least, I've not seen the log perm errors since I made the change; _do_ need to watch for a bit.

I'd cribbed the .service from posts online, so don't have full confidence in the hardening settings yet.

Is there, generally, any expected harm/risk to a Postfix setup's security/perms model with that^ change,

- NoNewPrivileges=yes
+ NoNewPrivileges=no

that should better be handled in Postfix configs?

[Viktor Dukhovni]

On Wed, Oct 28, 2020 at 09:01:38AM -0700, PGNet Dev wrote:

> Oct 28 15:02:40 svr019 postfix/postdrop[64624]: warning: mail_queue_enter: create file maildrop/553726.64624: Permission denied
> Oct 28 15:02:45 svr019 postfix/postdrop[32688]: warning: mail_queue_enter: create file maildrop/766615.32688: Permission denied

Barring interference from SELinux or AppArmour, ... this should not
happen unless file permissions change.  Correct file permissions are set
via "postfix set-permissions" (at package install time) and should not
change thereafter.

For reference, on my system:

    $ postconf setgid_group
    setgid_group = maildrop
    $ ls -ld /var/spool/postfix/maildrop
    drwx-wx---  2 postfix  maildrop  2 Oct 28 12:52 /var/spool/postfix/maildrop
    $ ls -l /usr/local/sbin/postdrop
    -rwxr-sr-x  1 root  maildrop  41656 Oct 25 03:44 /usr/local/sbin/postdrop

Provided that setgid bit and group of the postdrop executable
allows it to write to /var/spool/postfix/maildrop, and all
parent directories have "x" for all users, all should work.

If /, /var, /var/spool, /var/spool/postfix are not world-accessible
(x bit for "other"), or if /var/spool/maildrop is not group-wx, or
postdrop is not setgid, or has the wrong group, then things don't
work.

Of course if SELinux et. al, decide to intervene, then you have to
fix the relevant settings.

--
    Viktor.

[PGNet Dev]

On 10/28/20 10:00 AM, Viktor Dukhovni wrote:
> On Wed, Oct 28, 2020 at 09:01:38AM -0700, PGNet Dev wrote:
>
>> Oct 28 15:02:40 svr019 postfix/postdrop[64624]: warning: mail_queue_enter: create file maildrop/553726.64624: Permission denied
>> Oct 28 15:02:45 svr019 postfix/postdrop[32688]: warning: mail_queue_enter: create file maildrop/766615.32688: Permission denied
>
> Barring interference from SELinux or AppArmour,

neither in-place/enabled here atm

> ... this should not
> happen unless file permissions change.  Correct file permissions are set
> via "postfix set-permissions" (at package install time)

yup.  and, also tried re-exec'ing here.  so far, no curative effect by itself.

> and should not change thereafter.

that's what i've always seen to date.  and why i suspect it's something _else_ i've done that's monkeying with it.

> For reference, on my system:
>
>      $ postconf setgid_group
>      setgid_group = maildrop
>      $ ls -ld /var/spool/postfix/maildrop
>      drwx-wx---  2 postfix  maildrop  2 Oct 28 12:52 /var/spool/postfix/maildrop
>      $ ls -l /usr/local/sbin/postdrop
>      -rwxr-sr-x  1 root  maildrop  41656 Oct 25 03:44 /usr/local/sbin/postdrop

here (fwiw, this is Fedora32-packaged Postfix et al), minor diff

   postconf setgid_group
!!  setgid_group = postdrop

   ls -ld /var/spool/postfix/maildrop
     drwx-wx--- 2 postfix postdrop 4.0K Oct 28 08:45 /var/spool/postfix/maildrop/

   ls -l `which postdrop`
     -rwxr-sr-x 1 root postdrop 25K Aug 31 02:52 /usr/sbin/postdrop*

> Provided that setgid bit and group of the postdrop executable
> allows it to write to /var/spool/postfix/maildrop, and all
> parent directories have "x" for all users, all should work.

> If /, /var, /var/spool, /var/spool/postfix are not world-accessible
> (x bit for "other"), or if /var/spool/maildrop is not group-wx, or
> postdrop is not setgid, or has the wrong group, then things don't
> work.

as set by a relatively new install -- i.e., _not_ manually by me,



   ls -ald / /var /var/spool /var/spool/postfix

     dr-xr-xr-x. 22 root root 4.0K Sep  6 10:59 //

     drwxr-xr-x. 21 root root 4.0K Oct 27 20:36 /var/

     drwxr-xr-x  14 root root 4.0K Oct 25 19:05 /var/spool/

     drwxr-xr-x  16 root root 4.0K Aug 31 02:52 /var/spool/postfix/





so far, on this install, 'everything works' ... except for the eventual/random appearance of those^ complaints.

after changing that NoNewPerms bit in the pflogsumm.sh, I'm seeing no further errors.
now that I've clearly jinxed it ... i'll keep watch some more.

[Viktor Dukhovni]

On Wed, Oct 28, 2020 at 10:13:23AM -0700, PGNet Dev wrote:

> > For reference, on my system:
> >
> >      $ postconf setgid_group
> >      setgid_group = maildrop
> >      $ ls -ld /var/spool/postfix/maildrop
> >      drwx-wx---  2 postfix  maildrop  2 Oct 28 12:52 /var/spool/postfix/maildrop
> >      $ ls -l /usr/local/sbin/postdrop
> >      -rwxr-sr-x  1 root  maildrop  41656 Oct 25 03:44 /usr/local/sbin/postdrop
>
> here (fwiw, this is Fedora32-packaged Postfix et al), minor diff
>
>    postconf setgid_group
> !!  setgid_group = postdrop

The specific group name is not important.  It just has to be the
same group for the executable and the directory.

>    ls -ld /var/spool/postfix/maildrop
>      drwx-wx--- 2 postfix postdrop 4.0K Oct 28 08:45 /var/spool/postfix/maildrop/
>
>    ls -l `which postdrop`
>      -rwxr-sr-x 1 root postdrop 25K Aug 31 02:52 /usr/sbin/postdrop*

These look OK.

> as set by a relatively new install -- i.e., _not_ manually by me,
>
>    ls -ald / /var /var/spool /var/spool/postfix
>
>      dr-xr-xr-x. 22 root root 4.0K Sep  6 10:59 //
>      drwxr-xr-x. 21 root root 4.0K Oct 27 20:36 /var/
>      drwxr-xr-x  14 root root 4.0K Oct 25 19:05 /var/spool/
>      drwxr-xr-x  16 root root 4.0K Aug 31 02:52 /var/spool/postfix/

That "x." suggests some extended POSIX ACLs on / and /var, but they're
likely OK.  You can double-check these.

Otherwise, you can check that no kernel settings disable setgid
execution.

> so far, on this install, 'everything works' ... except for the
> eventual/random appearance of those^ complaints.

You'll need to figure why the EPERM happens.  Postfix is just
the messenger.

--
    Viktor.

[Bastian Blank-3]

Hi Viktor

On Wed, Oct 28, 2020 at 01:00:35PM -0400, Viktor Dukhovni wrote:
> On Wed, Oct 28, 2020 at 09:01:38AM -0700, PGNet Dev wrote:
> > Oct 28 15:02:40 svr019 postfix/postdrop[64624]: warning: mail_queue_enter: create file maildrop/553726.64624: Permission denied
> > Oct 28 15:02:45 svr019 postfix/postdrop[32688]: warning: mail_queue_enter: create file maildrop/766615.32688: Permission denied
> Barring interference from SELinux or AppArmour, ... this should not
> happen unless file permissions change.

Maybe this was true ten years ago, but it is not longer.  The OP even
mentioned something called "no new privileges", which is described only
one tiny internet search away at
https://www.kernel.org/doc/html/latest/userspace-api/no_new_privs.html.

And yes, this flag is exactly what can cause this: it disables
suid/sgid, so the maildrop process runs without it and of course can't
write into the maildrop directory.

>                                         Correct file permissions are set
> via "postfix set-permissions" (at package install time) and should not
> change thereafter.

The OP even showed that the permissions are correct, but you did not
cite it.

> Of course if SELinux et. al, decide to intervene, then you have to
> fix the relevant settings.

It seems that you have to learn a lot about the security controls that
a modern Linux provides.

Regards,
Bastian

--
I object to intellect without discipline;  I object to power without
constructive purpose.
                -- Spock, "The Squire of Gothos", stardate 2124.5

[Viktor Dukhovni]

On Wed, Oct 28, 2020 at 06:19:10PM +0100, Bastian Blank wrote:

> > Barring interference from SELinux or AppArmour, ... this should not
> > happen unless file permissions change.
>
> Maybe this was true ten years ago, but it is not longer.  The OP even
> mentioned something called "no new privileges", which is described only
> one tiny internet search away at
> https://www.kernel.org/doc/html/latest/userspace-api/no_new_privs.html.

The "..." after "SELinux or AppArmour" is inclusive of other controls
that modify expected POSIX semantics.

Indeed a process with "no_new_privs" will not be able to run sendmail(1)
to submit new email.

> It seems that you have to learn a lot about the security controls that
> a modern Linux provides.

A less patronising tone would be welcome...

--
    Viktor.

[PGNet Dev]

On 10/28/20 10:32 AM, Viktor Dukhovni wrote:
> Indeed a process with "no_new_privs" will not be able to run sendmail(1)
> to submit new email.

noted.

that said, this _just_ reappeared here,

   postfix/postdrop[15673]: warning: mail_queue_enter: create file maildrop/678088.15673: Permission denied

so _not_ (just) pflogsumm :-/

ack that Postfix is just the msngr.

now to find what _other_ process is triggering this!

[Viktor Dukhovni]

On Wed, Oct 28, 2020 at 11:22:55AM -0700, PGNet Dev wrote:

> On 10/28/20 10:32 AM, Viktor Dukhovni wrote:
> > Indeed a process with "no_new_privs" will not be able to run sendmail(1)
> > to submit new email.
>
> noted.
>
> that said, this _just_ reappeared here,
>
>    postfix/postdrop[15673]: warning: mail_queue_enter: create file maildrop/678088.15673: Permission denied
>
> so _not_ (just) pflogsumm :-/
>
> ack that Postfix is just the msngr.
>
> now to find what _other_ process is triggering this!

You might start with:

    # grep -r NoNewPrivileges /etc/systemd

and all other directories with systemd unit files.

--
    Viktor.

[PGNet Dev]

On 10/28/20 11:30 AM, Viktor Dukhovni wrote:
> You might start with:
>
>      # grep -r NoNewPrivileges /etc/systemd
>
> and all other directories with systemd unit files.

yup.  already done.

nothing --other than the now "=false" (need to double check if that's the same as _removing_ it ) in pflogsumm.service -- under /etc/systemd

there _are_ the standard instances in /usr/lib/systemd/system/systemd-*, but none that seem immediately relevant.

i know I've done this "to myself" with the install; i've just _never_ had the problem with any DIY-installed postfix*.

so, atm, elusive :-/

[PGNet Dev]

On 10/28/20 11:36 AM, PGNet Dev wrote:
> On 10/28/20 11:30 AM, Viktor Dukhovni wrote:
>> You might start with:
>>
>>      # grep -r NoNewPrivileges /etc/systemd

i couldn't find any direct, relevant postdrop/maildrop, or NoNewPrivileges, references i chased sendmail usage instances instead.

i've clamav-milter in my milter chain on this box.

my clamav-milter.conf includes

        VirusAction /usr/local/etc/clamav/scripts/virus-alert.sh

where that script _does_ invoke sendmail.

found this process

        ps ax | grep virus
                15670 ?        S      0:00 /bin/bash /usr/local/etc/clamav/scripts/virus-alert.sh Sanesecurity.Jurlbl.Auto.85a586.UNOFFICIAL 4DLaEC42Q6z2Q [hidden email] [hidden email] Engineer Tee Shirt & Science Tee Shirt <[hidden email]> Tue, 27 Oct 2020 17:29:06 -0700

hanging around, obviously dealing with a spammy "Engineer Tee Shirt & Science Tee Shirt" send.

i killed it & verified that the current stream of

        warning: mail_queue_enter: create file maildrop/731085.15673: Permission denied

in postfix logs indeed stops.

so, seems like a likely bet!

clamav-milter does NOT have the

        NoNewPermissions=true

set, and therefore defaults to

        NoNewPermissions=false


That^ 'virus-alert.sh' -- exec'ing 'via' clamav-milter, obviously included in my postfix config -- needs to be given appropriate permissions for sending via sendmail.postfix.

The question is:

     what/where is the correct method, so as NOT to break my postfix security model in some foolish manner ?

[Wietse Venema]

PGNet Dev:

> my clamav-milter.conf includes
>
> VirusAction /usr/local/etc/clamav/scripts/virus-alert.sh
>
> where that script _does_ invoke sendmail.
>
> found this process
>
> ps ax | grep virus
> 15670 ?        S      0:00 /bin/bash /usr/local/etc/clamav/scripts/virus-alert.sh Sanesecurity.Jurlbl.Auto.85a586.UNOFFICIAL 4DLaEC42Q6z2Q [hidden email] [hidden email] Engineer Tee Shirt & Science Tee Shirt <[hidden email]> Tue, 27 Oct 2020 17:29:06 -0700
>
> hanging around, obviously dealing with a spammy "Engineer Tee Shirt & Science Tee Shirt" send.
>
> i killed it & verified that the current stream of
>
> warning: mail_queue_enter: create file maildrop/731085.15673: Permission denied
>
> in postfix logs indeed stops.
>
> so, seems like a likely bet!

One possible way out is to skip the Postfix sendmail command, and
to use a "mini sendmail" program that submits mail via SMTP.
Obviously that will fail when Postfix is down.

It is theoretically possible to build a 'submit via authenticated
SMTP' mode into the Postfix sendmail command, and/or to save
unsubmitted mail to ~/dead.letter, but both add complexity (new
bugs) and introduce new failure modes (more bugs).

        Wietse

[PGNet Dev]

On 10/28/20 2:38 PM, Wietse Venema wrote:
> One possible way out is to skip the Postfix sendmail command, and
> to use a "mini sendmail" program that submits mail via SMTP.

i've typically got msmtp rattling around.

> Obviously that will fail when Postfix is down.

noted.

not ideal, but not critical.

> It is theoretically possible to build a 'submit via authenticated
> SMTP' mode into the Postfix sendmail command, and/or to save
> unsubmitted mail to ~/dead.letter, but both add complexity (new
> bugs) and introduce new failure modes (more bugs).

i generally don't like the idea of adding more failure points.

i'm working to remove as much cruft as possible.  adding the notification script already is counterproductive to that goal.

seems to me there should be a way to give that extra script -- called by a service milter that _itself_ seems to have all its needed perms for its 'normal' ops -- the correct rights & perms to talk to sendmail.
from _outside_ of postfix.

which seems like not-a-postfix matter.

[PGNet Dev]

On 10/28/20 2:38 PM, Wietse Venema wrote:
> One possible way out is to skip the Postfix sendmail command, and
> to use a "mini sendmail" program that submits mail via SMTP.

adding an msmtp sender as the VirusAction script in clamav milter, though a bit of 'extra', certainly is the simplest.

easy to implement, and seems to behave under my testing -- no more perms errors.

will watch for awhile.