Problem with filter

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Problem with filter

Paul Schmehl-3
Years ago I setup postfix using the filter script that Wietse wrote to
integrate spamassassin into my mail setup. It's worked fine for a long time.

smtp      inet  n       -       n       -       -       smtpd -o
content_filter=filter:dummyr

On Monday, I updated the server from freebsd-10.4-RELEASE to 11.2-RELEASE.
This included a reinstall of all ports, since it was a major version
upgrade.

Now I'm getting very strange errors.

In the logs:

Dec 10 22:08:39 mail postfix/pipe[83221]: fatal: get_service_attr: unknown
username: filter
Dec 10 22:08:40 mail postfix/qmgr[25686]: warning: private/filter socket:
malformed response
Dec 10 22:08:40 mail postfix/qmgr[25686]: warning: transport filter failure
-- see a previous warning/fatal/panic logfile record for the problem
description
Dec 10 22:08:40 mail postfix/master[25684]: warning: process
/usr/local/libexec/postfix/pipe pid 83221 exit status 1
Dec 10 22:08:40 mail postfix/master[25684]: warning:
/usr/local/libexec/postfix/pipe: bad command startup -- throttling

As a result, mail is not being delivered to courier but is piling up in the
queue (except for internal mail, of course).

The problem is, filter IS a legitimate user:

# grep filter /etc/passwd
filter:*:1004:1004:User &:/home/filter:/bin/sh

And the passwd file is world-readable:

# ls -lsa /etc/passwd
4 -rw-r--r--  1 root  wheel  2931 Dec 10 22:04 /etc/passwd

So why does postfix think this user does not exist?

The log entry above is the first instance of the error, and I believe all
errors are logged to /var/log/maillog. I've verified that filter's home
directory exists and is owned by him as well as the directory where filter
puts emails that are detected to be spam.

There is one anomaly in his home directory that strikes me as a possible
clue.

 ls -lsa /home/filter/
total 24
2 drwxr-xr-x   4 1004  filter  512 Sep 27  2012 .
2 drwxr-xr-x  13 root  wheel   512 Nov 19  2017 ..
2 -rw-r--r--   1 1004  filter  751 Sep 27  2012 .cshrc
2 -rw-r--r--   1 1004  filter  248 Sep 27  2012 .login
2 -rw-r--r--   1 1004  filter  158 Sep 27  2012 .login_conf
2 -rw-------   1 1004  filter  373 Sep 27  2012 .mail_aliases
2 -rw-r--r--   1 1004  filter  331 Sep 27  2012 .mailrc
2 -rw-r--r--   1 1004  filter  766 Sep 27  2012 .profile
2 -rw-------   1 1004  filter  276 Sep 27  2012 .rhosts
2 -rw-r--r--   1 1004  filter  975 Sep 27  2012 .shrc
2 drwx------   2 1004  filter  512 Dec 10 21:45 .spamassassin
2 drwx------   5 1004  filter  512 Sep 27  2012 Maildir

Note that the .spamassassin folder was created on Dec 10, the day of the
upgrade. Everything else dates back to the origin of the directory.
Everything spamassassin-related used to be in
/usr/local/etc/mail/spamassassin, including bayes_seen, and all of those
files are owned by filter as well.

The following items are in that folder:

# ls -lsa /home/filter/.spamassassin/
total 25056
    2 drwx------  2 1004  filter       512 Dec 10 21:45 .
    2 drwxr-xr-x  4 1004  filter       512 Sep 27  2012 ..
 2592 -rw-------  1 1004  filter   2633728 Sep 27  2012 auto-whitelist
   12 -rw-------  1 1004  filter     11448 Dec 10 21:58 bayes_journal
18304 -rw-------  1 1004  filter  20299776 Dec 10 21:45 bayes_seen
 4144 -rw-------  1 1004  filter   5193728 Dec 10 21:45 bayes_toks
    0 -rw-r--r--  1 1004  filter         0 Dec 26  2012 user_prefs

I'm at a loss. It's probably something obvious, but I don't see it.

Paul Schmehl
Independent Researcher
Reply | Threaded
Open this post in threaded view
|

Re: Problem with filter

Benny Pedersen-2
Paul Schmehl skrev den 2018-12-13 18:36:
> # ls -lsa /home/filter/.spamassassin/
> total 25056
>    2 drwx------  2 1004  filter       512 Dec 10 21:45 .

why is 1004 when it should be ascii usernamme ?

what user is 1004 ?, its imho not known in your system

id 1004
Reply | Threaded
Open this post in threaded view
|

Re: Problem with filter

Paul Schmehl-3
--On December 13, 2018 at 6:54:48 PM +0100 Benny Pedersen <[hidden email]>
wrote:

> Paul Schmehl skrev den 2018-12-13 18:36:
>> # ls -lsa /home/filter/.spamassassin/
>> total 25056
>>    2 drwx------  2 1004  filter       512 Dec 10 21:45 .
>
> why is 1004 when it should be ascii usernamme ?
>
> what user is 1004 ?, its imho not known in your system
>
> id 1004

1004 is the uid of filter. It shouldn't make a difference since the ascii
name and uid refer to the same account.

Paul Schmehl
Independent Researcher
Reply | Threaded
Open this post in threaded view
|

Re: Problem with filter

Viktor Dukhovni
> On Dec 13, 2018, at 2:33 PM, Paul Schmehl <[hidden email]> wrote:
>
>>> # ls -lsa /home/filter/.spamassassin/
>>> total 25056
>>>   2 drwx------  2 1004  filter       512 Dec 10 21:45 .
>>
>> why is 1004 when it should be ascii usernamme ?
>>
>> what user is 1004 ?, its imho not known in your system
>>
>> id 1004
>
> 1004 is the uid of filter. It shouldn't make a difference since the ascii name and uid refer to the same account.

It makes all the difference, since "ls -l" shows that the uid does not
in fact resolve to that user name.  Perhaps there's a syntax error
earlier in your passwd file?  And perhaps the user is missing from the
"shadow" password file...

The user does not exist until "ls -l" is able to correctly identify the
files as belonging to the user.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: Problem with filter

Paul Schmehl-3
--On December 13, 2018 at 2:36:44 PM -0500 Viktor Dukhovni
<[hidden email]> wrote:

>> On Dec 13, 2018, at 2:33 PM, Paul Schmehl <[hidden email]> wrote:
>>
>>>> # ls -lsa /home/filter/.spamassassin/
>>>> total 25056
>>>>   2 drwx------  2 1004  filter       512 Dec 10 21:45 .
>>>
>>> why is 1004 when it should be ascii usernamme ?
>>>
>>> what user is 1004 ?, its imho not known in your system
>>>
>>> id 1004
>>
>> 1004 is the uid of filter. It shouldn't make a difference since the
>> ascii name and uid refer to the same account.
>
> It makes all the difference, since "ls -l" shows that the uid does not
> in fact resolve to that user name.  Perhaps there's a syntax error
> earlier in your passwd file?  And perhaps the user is missing from the
> "shadow" password file...
>
> The user does not exist until "ls -l" is able to correctly identify the
> files as belonging to the user.

Hmmm...thank you, Victor. I'll try to sort that out.

Paul Schmehl
Independent Researcher
Reply | Threaded
Open this post in threaded view
|

Re: Problem with filter

Benny Pedersen-2
Paul Schmehl skrev den 2018-12-13 20:45:

>> The user does not exist until "ls -l" is able to correctly identify
>> the
>> files as belonging to the user.
>
> Hmmm...thank you, Victor. I'll try to sort that out.

if its more simple, change to use spampd so spamassassin is smtp content
filter seen from postfix

its have being rock solid for me, and i only use clamav-milter for virus
scanning, and opendmarc, opendkim, spf-policyd, thats all i need :=)
Reply | Threaded
Open this post in threaded view
|

Re: Problem with filter

Paul Schmehl-3
--On December 13, 2018 at 9:00:06 PM +0100 Benny Pedersen <[hidden email]>
wrote:

> Paul Schmehl skrev den 2018-12-13 20:45:
>
>>> The user does not exist until "ls -l" is able to correctly identify
>>> the
>>> files as belonging to the user.
>>
>> Hmmm...thank you, Victor. I'll try to sort that out.
>
> if its more simple, change to use spampd so spamassassin is smtp content
> filter seen from postfix
>
> its have being rock solid for me, and i only use clamav-milter for virus
> scanning, and opendmarc, opendkim, spf-policyd, thats all i need :=)

That's what I've been doing, but apparently my /etc/passwd file is screwed
up.

Paul Schmehl
Independent Researcher
Reply | Threaded
Open this post in threaded view
|

Re: Problem with filter

@lbutlr


> On 13 Dec 2018, at 20:05, Paul Schmehl <[hidden email]> wrote:
>
> --On December 13, 2018 at 9:00:06 PM +0100 Benny Pedersen <[hidden email]> wrote:
>
>> Paul Schmehl skrev den 2018-12-13 20:45:
>>
>>>> The user does not exist until "ls -l" is able to correctly identify
>>>> the
>>>> files as belonging to the user.
>>>
>>> Hmmm...thank you, Victor. I'll try to sort that out.
>>
>> if its more simple, change to use spampd so spamassassin is smtp content
>> filter seen from postfix
>>
>> its have being rock solid for me, and i only use clamav-milter for virus
>> scanning, and opendmarc, opendkim, spf-policyd, thats all i need :=)
>
> That's what I've been doing, but apparently my /etc/passwd file is screwed up.

I had a similar issue when I moved from 10.x to 11.x where a user account line in the passwd field was mangled in some minor way and it cause the rest of the passed file to not be processed.

I do not recall the details, but I think on UID was changed from something like 1015 to 1000015?

So, look at the passwd file and move the ‘filter’ user earlier in the file, If that fixes it, you can then use the id command to check each UID later in the file to narrow down where the problem is.

--
"A politician is a man who approaches every problem with an open mouth.”


Reply | Threaded
Open this post in threaded view
|

Re: Problem with filter

Paul Schmehl-3
--On December 14, 2018 at 2:49:36 PM -0700 "@lbutlr" <[hidden email]>
wrote:

>
>
>> On 13 Dec 2018, at 20:05, Paul Schmehl <[hidden email]> wrote:
>>
>> --On December 13, 2018 at 9:00:06 PM +0100 Benny Pedersen <[hidden email]>
>> wrote:
>>
>>> Paul Schmehl skrev den 2018-12-13 20:45:
>>>
>>>>> The user does not exist until "ls -l" is able to correctly identify
>>>>> the
>>>>> files as belonging to the user.
>>>>
>>>> Hmmm...thank you, Victor. I'll try to sort that out.
>>
[snipped]

>> That's what I've been doing, but apparently my /etc/passwd file is
>> screwed up.
>
> I had a similar issue when I moved from 10.x to 11.x where a user account
> line in the passwd field was mangled in some minor way and it cause the
> rest of the passed file to not be processed.
>

I had to run fsck twice to get the system back up and running.

> I do not recall the details, but I think on UID was changed from
> something like 1015 to 1000015?
>
> So, look at the passwd file and move the ‘filter’ user earlier in the
> file, If that fixes it, you can then use the id command to check each UID
> later in the file to narrow down where the problem is.

It's fixed now. I ran pwd_mkdb -C and it reported that the file was
corrupted. Which is pretty useless, because it says the same thing on a
perfectly fine passwd file.

At any rate, I rebuilt it, and that did the trick.

Paul Schmehl
Independent Researcher