dnsblog lifetime

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

dnsblog lifetime

wa6vvv
I understood from the dnsblog man page that each dnsblog process only lives for a "limited amount of time".  I noticed this because I have over 50 dnsblog processes running on a fairly light duty postfix server.  Some of them are over a week old.  At first I thought they must have been orphaned, but looking through maillog, I find entries in the last few minutes from the oldest and the newest.  I didn't check all of them, but it appears they are all in use.  Looking at the source for postfix-3.3-20180114 (on web), it appears dnsblog checks one IP address and then exits.  I believe I can limit the number of dnsblog processes in master.cf (currently set to 0), but I am not sure that is a good idea.  How long are these processes supposed to live?

-- Doug

Reply | Threaded
Open this post in threaded view
|

Re: dnsblog lifetime

Wietse Venema
Doug Hardie:

> I understood from the dnsblog man page that each dnsblog process
> only lives for a "limited amount of time".  I noticed this because
> I have over 50 dnsblog processes running on a fairly light duty
> postfix server.  Some of them are over a week old.  At first I
> thought they must have been orphaned, but looking through maillog,
> I find entries in the last few minutes from the oldest and the
> newest.  I didn't check all of them, but it appears they are all
> in use.  Looking at the source for postfix-3.3-20180114 (on web),
> it appears dnsblog checks one IP address and then exits.  I believe
> I can limit the number of dnsblog processes in master.cf (currently
> set to 0), but I am not sure that is a good idea.  How long are
> these processes supposed to live?

According to source, dnsblog processes exclude themselves from the
max_use limit (max_idle remains in effect). I suppose I turned off
max_use because these processes are postscreen helpers. Postscreen
was designed to handle a much larger client load than to the rest
of Postfix. Under extreme loads like 10000+ connections/second,
one does not want to be creating 100+ processes/second, as that
would limit scalability.

The dnsblog processes still terminate after 100s idle time. On my
lightly-loaded server, there currently is no dnsblog process running.

Apparently your server has enough traffic to keep postscreen alive,
and as a consequence, a collection of dnsblog processes.

I suppose you could reduce max_idle, but don't go overboard and
set it to something small like 1s. That would be counterproductive.

        Wiemaketse
Reply | Threaded
Open this post in threaded view
|

Re: dnsblog lifetime

wa6vvv
> On 22 April 2018, at 05:50, Wietse Venema <[hidden email]> wrote:
>
> Doug Hardie:
>> I understood from the dnsblog man page that each dnsblog process
>> only lives for a "limited amount of time".  I noticed this because
>> I have over 50 dnsblog processes running on a fairly light duty
>> postfix server.  Some of them are over a week old.  At first I
>> thought they must have been orphaned, but looking through maillog,
>> I find entries in the last few minutes from the oldest and the
>> newest.  I didn't check all of them, but it appears they are all
>> in use.  Looking at the source for postfix-3.3-20180114 (on web),
>> it appears dnsblog checks one IP address and then exits.  I believe
>> I can limit the number of dnsblog processes in master.cf (currently
>> set to 0), but I am not sure that is a good idea.  How long are
>> these processes supposed to live?
>
> According to source, dnsblog processes exclude themselves from the
> max_use limit (max_idle remains in effect). I suppose I turned off
> max_use because these processes are postscreen helpers. Postscreen
> was designed to handle a much larger client load than to the rest
> of Postfix. Under extreme loads like 10000+ connections/second,
> one does not want to be creating 100+ processes/second, as that
> would limit scalability.
>
> The dnsblog processes still terminate after 100s idle time. On my
> lightly-loaded server, there currently is no dnsblog process running.
>
> Apparently your server has enough traffic to keep postscreen alive,
> and as a consequence, a collection of dnsblog processes.
>
> I suppose you could reduce max_idle, but don't go overboard and
> set it to something small like 1s. That would be counterproductive.

Thanks for the info.  I never would have expected my server to be that busy.  I would suggest that the man page for dnsblog be updated in the first paragraph of configuration parameters to something like:

Changes to main.cf are picked up automatically as the dnsblog processes are started.  The dnsblog processes terminate after 100 seconds of idle time.  That is the default value and can be changed with the max idle configuration parameter.  However, reducing it to a small value is likely to be counterproductive.  Use the command "postfix reload" to speed up a change.

-- Doug


Reply | Threaded
Open this post in threaded view
|

Re: dnsblog lifetime

Wietse Venema
Doug Hardie:

> > On 22 April 2018, at 05:50, Wietse Venema <[hidden email]> wrote:
> >
> > Doug Hardie:
> >> I understood from the dnsblog man page that each dnsblog process
> >> only lives for a "limited amount of time".  I noticed this because
> >> I have over 50 dnsblog processes running on a fairly light duty
> >> postfix server.  Some of them are over a week old.  At first I
> >> thought they must have been orphaned, but looking through maillog,
> >> I find entries in the last few minutes from the oldest and the
> >> newest.  I didn't check all of them, but it appears they are all
> >> in use.  Looking at the source for postfix-3.3-20180114 (on web),
> >> it appears dnsblog checks one IP address and then exits.  I believe
> >> I can limit the number of dnsblog processes in master.cf (currently
> >> set to 0), but I am not sure that is a good idea.  How long are
> >> these processes supposed to live?
> >
> > According to source, dnsblog processes exclude themselves from the
> > max_use limit (max_idle remains in effect). I suppose I turned off
> > max_use because these processes are postscreen helpers. Postscreen
> > was designed to handle a much larger client load than to the rest
> > of Postfix. Under extreme loads like 10000+ connections/second,
> > one does not want to be creating 100+ processes/second, as that
> > would limit scalability.
> >
> > The dnsblog processes still terminate after 100s idle time. On my
> > lightly-loaded server, there currently is no dnsblog process running.

I think that we can avoid the need for warnings in documentation,
by making the dnsblog service act according to the spirit of the
max_idle and max_use settings, even if it cannot act by the letter.

With a given max_idle and max_use setting, a process is expected
to terminate within approximately (max_idle * max_use) seconds.
That is, on a low-volume (but not too low) server, a process may
hang around for a few hours (100*100 = 10000 seconds).

Even if the dnsblog process cannot enforce max_use literally (because
dbsnlog may have to handle a huge number of requests during peak
load), the process could still retire voluntarily after (max_idle
* max_use) seconds, without any negative performance impact.

I'll look into implementing that.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: dnsblog lifetime

wa6vvv
> On 24 April 2018, at 13:48, Wietse Venema <[hidden email]> wrote:
>
> Doug Hardie:
>>> On 22 April 2018, at 05:50, Wietse Venema <[hidden email]> wrote:
>>>
>>> Doug Hardie:
>>>> I understood from the dnsblog man page that each dnsblog process
>>>> only lives for a "limited amount of time".  I noticed this because
>>>> I have over 50 dnsblog processes running on a fairly light duty
>>>> postfix server.  Some of them are over a week old.  At first I
>>>> thought they must have been orphaned, but looking through maillog,
>>>> I find entries in the last few minutes from the oldest and the
>>>> newest.  I didn't check all of them, but it appears they are all
>>>> in use.  Looking at the source for postfix-3.3-20180114 (on web),
>>>> it appears dnsblog checks one IP address and then exits.  I believe
>>>> I can limit the number of dnsblog processes in master.cf (currently
>>>> set to 0), but I am not sure that is a good idea.  How long are
>>>> these processes supposed to live?
>>>
>>> According to source, dnsblog processes exclude themselves from the
>>> max_use limit (max_idle remains in effect). I suppose I turned off
>>> max_use because these processes are postscreen helpers. Postscreen
>>> was designed to handle a much larger client load than to the rest
>>> of Postfix. Under extreme loads like 10000+ connections/second,
>>> one does not want to be creating 100+ processes/second, as that
>>> would limit scalability.
>>>
>>> The dnsblog processes still terminate after 100s idle time. On my
>>> lightly-loaded server, there currently is no dnsblog process running.
>
> I think that we can avoid the need for warnings in documentation,
> by making the dnsblog service act according to the spirit of the
> max_idle and max_use settings, even if it cannot act by the letter.
>
> With a given max_idle and max_use setting, a process is expected
> to terminate within approximately (max_idle * max_use) seconds.
> That is, on a low-volume (but not too low) server, a process may
> hang around for a few hours (100*100 = 10000 seconds).
>
> Even if the dnsblog process cannot enforce max_use literally (because
> dbsnlog may have to handle a huge number of requests during peak
> load), the process could still retire voluntarily after (max_idle
> * max_use) seconds, without any negative performance impact.
>
> I'll look into implementing that.

Either way works for me.  I just got confused when I saw the durations of the processes and then read the man page.  I thought I had configured something incorrectly as they didn't match.  If I had your first response in the man page I would have said ahhh, now I understand.  

-- Doug