Recommended way to (quickly) get total mail queue size?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Recommended way to (quickly) get total mail queue size?

Michael Durket-2
In situations where there are tens of thousands of messages piling up in
the Postfix mail queue, running a 'postqueue -p' command and then
waiting until the last line prints isn't a very timely way to find out  
the
current queue size (I'm thinking in terms of a periodic monitor like
Nagios for example).

So what's the best way to quickly (i.e. less than a few seconds) get the
current queue count out of Postfix?

Reply | Threaded
Open this post in threaded view
|

Re: Recommended way to (quickly) get total mail queue size?

Wietse Venema
Michael Durket:
> In situations where there are tens of thousands of messages piling up in
> the Postfix mail queue, running a 'postqueue -p' command and then
> waiting until the last line prints isn't a very timely way to find out  
> the
> current queue size (I'm thinking in terms of a periodic monitor like
> Nagios for example).
>
> So what's the best way to quickly (i.e. less than a few seconds) get the
> current queue count out of Postfix?

Fastest: have a dedicated file system and count the number
of used versus free inodes (df -i). This produces an
instantaneous result regardless of the queue size (*).

Slower: a suitable perl script that exploits the fact that
directories have one-character names. I'll leave that up
to Victor.

Slower: a suitable incantation of the find command.

Slowest: mailq/postqueue -p, because that opens each file.

        Wietse

(*) Except on systems where df(1) secretly invokes sync(2);
    SunOS was such an example.
Reply | Threaded
Open this post in threaded view
|

Re: Recommended way to (quickly) get total mail queue size?

Trey Briggs-2
On Tue, Jul 7, 2009 at 10:47 AM, Wietse Venema <[hidden email]> wrote:
Slower: a suitable incantation of the find command.

Here's that one:

#!/bin/sh
qdir=`postconf -h queue_directory`
incoming=`find $qdir/incoming -type f -print | wc -l | awk '{print $1}'`
activeonly=`find $qdir/active -type f -print | wc -l | awk '{print $1}'`
maildrop=`find $qdir/maildrop -type f -print | wc -l | awk '{print $1}'`
active=`find $qdir/incoming $qdir/active $qdir/maildrop -type f -print | wc -l | awk '{print $1}'`
deferred=`find $qdir/deferred -type f -print | wc -l | awk '{print $1}'`
printf "active: %d\ndeferred: %d\nincoming: %d\nactiveonly: %d\nmaildrop: %d\n" $active $deferred $incoming $activeonly $maildrop

--
Trey Briggs
Advanced Systems Engineer
API Digital
Reply | Threaded
Open this post in threaded view
|

Re: Recommended way to (quickly) get total mail queue size?

Simon Waters
In reply to this post by Michael Durket-2
On Tuesday 07 July 2009 16:15:06 Michael Durket wrote:
>
> So what's the best way to quickly (i.e. less than a few seconds) get the
> current queue count out of Postfix?

man qshape
Reply | Threaded
Open this post in threaded view
|

Re: Recommended way to (quickly) get total mail queue size?

Victor Duchovni
In reply to this post by Wietse Venema
On Tue, Jul 07, 2009 at 11:47:51AM -0400, Wietse Venema wrote:

> Michael Durket:
> > In situations where there are tens of thousands of messages piling up in
> > the Postfix mail queue, running a 'postqueue -p' command and then
> > waiting until the last line prints isn't a very timely way to find out  
> > the
> > current queue size (I'm thinking in terms of a periodic monitor like
> > Nagios for example).
> >
> > So what's the best way to quickly (i.e. less than a few seconds) get the
> > current queue count out of Postfix?
>
> Fastest: have a dedicated file system and count the number
> of used versus free inodes (df -i). This produces an
> instantaneous result regardless of the queue size (*).
>
> Slower: a suitable perl script that exploits the fact that
> directories have one-character names. I'll leave that up
> to Victor.

Code along these lines has been posted before, I believe:

    # perl -e '
        use strict;
        use warnings;
        use Symbol;
        sub count {
            my ($dir) = @_;
            my $dh = gensym();
            my $c = 0;
            opendir($dh, $dir) or die "$0: opendir: $dir: $!\n";
            while (my $f = readdir($dh)) {
                if ($f =~ m{^[A-F0-9]{5,}$}) {
                    ++$c;
                } elsif ($f =~ m{^[A-F0-9]$}) {
                    $c += count("$dir/$f");
                }
            }
            closedir($dh) or die "closedir: $dir: $!\n";
            return $c;
        }
        my $qdir = shift(@ARGV) or die "Usage: $0 queue-directory\n";
        chdir($qdir) or die "$0: chdir: $qdir: $!\n";
        printf "Incoming: %d\n", count("incoming");
        printf "Active: %d\n", count("active");
        printf "Deferred: %d\n", count("deferred");
    ' /var/spool/postfix

This avoids all lstat(2) operations, reducing the system-call overhead
of counting queued messages to a minimum. The incoming queue number may
be a bit "inflated" because messages that are not yet fully received
are also counted). This overcount of the incoming queue is not generally
a problem.

--
        Viktor.

Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.

To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:[hidden email]?body=unsubscribe%20postfix-users>

If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.
Reply | Threaded
Open this post in threaded view
|

Re: Recommended way to (quickly) get total mail queue size?

Stuart Matthews
In reply to this post by Michael Durket-2
I just use:
mailq|grep Requests

- Stu

Michael Durket wrote:
> In situations where there are tens of thousands of messages piling up in
> the Postfix mail queue, running a 'postqueue -p' command and then
> waiting until the last line prints isn't a very timely way to find out the
> current queue size (I'm thinking in terms of a periodic monitor like
> Nagios for example).
>
> So what's the best way to quickly (i.e. less than a few seconds) get the
> current queue count out of Postfix?
>

Reply | Threaded
Open this post in threaded view
|

Re: Recommended way to (quickly) get total mail queue size?

Barney Desmond
2009/7/8 Stuart Matthews <[hidden email]>:
> I just use:
> mailq|grep Requests

Except that this method was expressly *not* desired because it's so
slow... (nb: mailq is a sendmail-compat feature that calls postqueue)

I'll be taking home some of these nice improvements, though. :)
Reply | Threaded
Open this post in threaded view
|

Re: Recommended way to (quickly) get total mail queue size?

Victor Duchovni
In reply to this post by Stuart Matthews
On Tue, Jul 07, 2009 at 10:19:23AM -0700, Stuart Matthews wrote:

> I just use:
> mailq|grep Requests

This does not reduce the I/O cost of reading all those queue files...
The OP requested an efficient method for large queues.

--
        Viktor.

Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.

To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:[hidden email]?body=unsubscribe%20postfix-users>

If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.
Reply | Threaded
Open this post in threaded view
|

Re: Recommended way to (quickly) get total mail queue size?

Bill Anderson-2
In reply to this post by Wietse Venema
>
> Slowest: mailq/postqueue -p, because that opens each file.


This would also be unwise given that mailq/postqueue -p can return  
incomplete results when the number queued files gets large - or more  
accurately that there is a limit to the amount of data mailq will show  
and thus it isn't specifically a message count limit.

On Linux systems I use an inotify enabled kernel (>= 2.6.13) and track  
the events via daemon/db. I then query the daemon when I need counts.  
You could do similarly on systems that provide an inotify-like feature.


Cheers,
Bill
Reply | Threaded
Open this post in threaded view
|

Re: Recommended way to (quickly) get total mail queue size?

Wietse Venema
Bill Anderson:
> >
> > Slowest: mailq/postqueue -p, because that opens each file.
>
>
> This would also be unwise given that mailq/postqueue -p can return  
> incomplete results when the number queued files gets large - or more  
> accurately that there is a limit to the amount of data mailq will show  
> and thus it isn't specifically a message count limit.

It's a time limit.  After $daemon_timeout (default: 18000s, or 5
hours) the mailq/postqueue queue listing will be aborted.

This is purely a safety feature that prevents filling up the machine
with cron jobs that query the queue status.

        Wietse

> On Linux systems I use an inotify enabled kernel (>= 2.6.13) and track  
> the events via daemon/db. I then query the daemon when I need counts.  
> You could do similarly on systems that provide an inotify-like feature.
>
>
> Cheers,
> Bill
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Recommended way to (quickly) get total mail queue size?

Barney Desmond
2009/7/8 Wietse Venema <[hidden email]>:
> It's a time limit.  After $daemon_timeout (default: 18000s, or 5
> hours) the mailq/postqueue queue listing will be aborted.

Alas, that won't save us. For monitoring purposes, you'd be checking
queue size every 5 *minutes* or so. And if our customers are anything
to go by, you can rely on them wanting to run cronjobs like that Every
Minute Of The Day, just to be Really, Really Sure. "What could
possibly go wrong?", etc.

That'd be one hell of a queue, but it's probably not as perverse an
occurrence as I imagine it to be.
Reply | Threaded
Open this post in threaded view
|

Re: Recommended way to (quickly) get total mail queue size?

Michael Durket-2
In reply to this post by Michael Durket-2
Thanks to all who replied. I'll be trying out the various ideas. But  
they all
seem awkward. In my MVS days, when we wrote large scale systems
like this, we'd keep important statistics in the link pack area,  
constantly
updated, so monitor programs could just map that area and get the
data (or easily create realtime displays for viewing throughout the
operations area). Since Postfix knows the queue size, and other  
relevant statistics,
how about doing something similar - creating a statistics block in  
global
memory that anyone can map to (readonly) and update queue information
and other data that might be useful there?

Reply | Threaded
Open this post in threaded view
|

Re: Recommended way to (quickly) get total mail queue size?

Victor Duchovni
On Wed, Jul 08, 2009 at 10:30:23AM -0700, Michael Durket wrote:

> Since Postfix knows the queue size, and other relevant
> statistics,

Postfix does not know the size of the deferred, incoming or maildrop
queues. This is intentional.

--
        Viktor.

Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.

To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:[hidden email]?body=unsubscribe%20postfix-users>

If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.
Reply | Threaded
Open this post in threaded view
|

Re: Recommended way to (quickly) get total mail queue size?

Wietse Venema
In reply to this post by Michael Durket-2
Michael Durket:
> Thanks to all who replied. I'll be trying out the various ideas. But  
> they all
> seem awkward. In my MVS days, when we wrote large scale systems
> like this, we'd keep important statistics in the link pack area,  
> constantly
> updated, so monitor programs could just map that area and get the
> data (or easily create realtime displays for viewing throughout the
> operations area). Since Postfix knows the queue size, and other  
> relevant statistics,

By design, Postfix does not know the size of the queue. All it
knows is how many messages are in the active queue, and that is a
limited subset of all mail.

        Wietse

> how about doing something similar - creating a statistics block in  
> global
> memory that anyone can map to (readonly) and update queue information
> and other data that might be useful there?
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Recommended way to (quickly) get total mail queue size?

@lbutlr
In reply to this post by Michael Durket-2
On Jul 7, 2009, at 9:15 AM, Michael Durket  
<[hidden email]> wrote:

> So what's the best way to quickly (i.e. less than a few seconds) get  
> the
> current queue count out of Postfix?

ls -1 [queue directory] | wc -l

?
Reply | Threaded
Open this post in threaded view
|

Re: Recommended way to (quickly) get total mail queue size?

Victor Duchovni
On Wed, Jul 08, 2009 at 12:44:17PM -0600, LuKreme wrote:

> On Jul 7, 2009, at 9:15 AM, Michael Durket <[hidden email]>
> wrote:
>
>> So what's the best way to quickly (i.e. less than a few seconds) get the
>> current queue count out of Postfix?
>
> ls -1 [queue directory] | wc -l

The Perl code I posted works when the queue directory is "hashed",
as is the case by default with:

        hash_queue_names = deferred, defer

(Note, "defer" does not store queued messages, just the associated
"bounce logs").

--
        Viktor.

Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.

To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:[hidden email]?body=unsubscribe%20postfix-users>

If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.
Reply | Threaded
Open this post in threaded view
|

Re: Recommended way to (quickly) get total mail queue size?

Bron Gondwana
In reply to this post by Wietse Venema
On Wed, Jul 08, 2009 at 02:00:49PM -0400, Wietse Venema wrote:

> Michael Durket:
> > Thanks to all who replied. I'll be trying out the various ideas. But  
> > they all
> > seem awkward. In my MVS days, when we wrote large scale systems
> > like this, we'd keep important statistics in the link pack area,  
> > constantly
> > updated, so monitor programs could just map that area and get the
> > data (or easily create realtime displays for viewing throughout the
> > operations area). Since Postfix knows the queue size, and other  
> > relevant statistics,
>
> By design, Postfix does not know the size of the queue. All it
> knows is how many messages are in the active queue, and that is a
> limited subset of all mail.

We use this - it's pretty disgusting, but it works fine.

#!/usr/bin/perl -w

use strict;
use warnings;
use Getopt::Std;

my %Opts;
getopts('da', \%Opts);

my $data = GetMqs($Opts{a} or $Opts{d}); # d implies all info!

print "$data->{total}\n";

if ($Opts{d}) {
  print << "__EOF";

Active:   $data->{active}
Held:     $data->{hold}
Incoming: $data->{incoming}
Deferred: $data->{deferred}
__EOF
}

sub GetMqs {
  my $gethold = shift;
  my $basedir = shift || "/var/spool/postfix";

  my $total = 0;
  my $active = 0;
  my $hold = 0;
  my $incoming = 0;
  my $deferred = 0;

  my @dirs = ('active', 'incoming', map { "deferred/$_" } 0..9, 'A'..'F');
  push @dirs, 'hold' if $gethold;
  foreach my $dir (@dirs) {
    opendir(DH, "/var/spool/postfix/$dir") || next;
    my $n = 0;
    while (my $item = readdir(DH)) {
      ++$n unless $item =~ m/^\./;
    }
    $total += $n;
    $active += $n if $dir eq 'active';
    $hold += $n if $dir eq 'hold';
    $incoming += $n if $dir eq 'incoming';
    $deferred += $n if substr($dir, 0, 8) eq 'deferred';
  }

  return {
    total => $total,
    active => $active,
    ($gethold ? (hold => $hold) : ()),
    incoming => $incoming,
    deferred => $deferred,
  };
}