Unexpected directories in virtual_mailbox_base

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Unexpected directories in virtual_mailbox_base

Thomas Seilund
Hi All,

I run a mail server with Postfix (version 2.6.6), Dovecot and Spamassassin.

The first time I saw an unexpected directory in virtual_mailbox_base
what medio dec. 2018. The mail server has been running for 5+ years.

There are more directories than the six directories I expect. I expect
one directory for each of the domains that the mail server handles.

Looking at the /var/log/maillog it seems as if:

1. An user that is handled by the mail server logs in and sends a mail
to an address that is not handled by the mail server

2. The mail is queued

3. Spamassassin kicks in and scans the mail. I don't understand that as
the mail is outgoing.

Perhaps Spamassassin creates the unexpected directory as the
Spamassassin line in /var/log/maillog refers to an unexpected file, ie.
/mnt/ebs01/vmail/landplan.dk/XXX/SpamAssassin/user_prefs. By the way,
that file does not exist.

Below is my postfix configuration

[ec2-user@ec2 ~]$ postconf -n
alias_maps = hash:/etc/aliases
broken_sasl_auth_clients = yes
config_directory = /etc/postfix
debug_peer_list = 85.191.189.106
disable_vrfy_command = yes
message_size_limit = 20480000
myhostname = ec2.netmaster.dk
recipient_bcc_maps = hash:/etc/postfix/recipient_bcc
smtp_tls_loglevel = 1
smtp_use_tls = yes
smtpd_client_restrictions =
smtpd_helo_required = yes
smtpd_helo_restrictions =
smtpd_recipient_restrictions = permit_mynetworks,
permit_sasl_authenticated,    reject_unauth_destination,
reject_invalid_hostname,    reject_unauth_pipelining,
reject_non_fqdn_sender,    reject_unknown_sender_domain,
reject_non_fqdn_recipient,    reject_unknown_recipient_domain, permit
smtpd_sasl_auth_enable = yes
smtpd_sasl_authenticated_header = yes
smtpd_sasl_local_domain =
smtpd_sasl_path = private/auth
smtpd_sasl_security_options = noanonymous
smtpd_sasl_type = dovecot
smtpd_sender_restrictions =
smtpd_tls_auth_only = yes
smtpd_tls_cert_file = /etc/letsencrypt/live/mail.netmaster.dk/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/mail.netmaster.dk/privkey.pem
smtpd_tls_loglevel = 1
smtpd_use_tls = yes
strict_rfc821_envelopes = yes
virtual_alias_maps = mysql:/etc/postfix/mysql/virtual_alias_maps.cf
virtual_gid_maps = static:2000
virtual_mailbox_base = /mnt/ebs01/vmail
virtual_mailbox_domains = mysql:/etc/postfix/mysql/virtual_domains_maps.cf
virtual_mailbox_maps = mysql:/etc/postfix/mysql/virtual_mailbox_maps.cf
virtual_minimum_uid = 2000
virtual_transport = lmtp:unix:private/dovecot-lmtp
virtual_uid_maps = static:2000
[ec2-user@ec2 ~]$

Any help would be appreciated

Thomas S

Reply | Threaded
Open this post in threaded view
|

Re: Unexpected directories in virtual_mailbox_base

John Fawcett
On 27/02/2019 17:56, Thomas Seilund wrote:

> Hi All,
>
> I run a mail server with Postfix (version 2.6.6), Dovecot and
> Spamassassin.
>
> The first time I saw an unexpected directory in virtual_mailbox_base
> what medio dec. 2018. The mail server has been running for 5+ years.
>
> There are more directories than the six directories I expect. I expect
> one directory for each of the domains that the mail server handles.

What were the directories that you did not expect?

>
> Looking at the /var/log/maillog it seems as if:
>
> 1. An user that is handled by the mail server logs in and sends a mail
> to an address that is not handled by the mail server
>
> 2. The mail is queued
>
> 3. Spamassassin kicks in and scans the mail. I don't understand that
> as the mail is outgoing.
You can scan outgoing mail if you want to make sure your users don't
send viruses. Even though I trust all my users I do it as a precaution
in case some users clients get infected and start sending viruses.

>
> Perhaps Spamassassin creates the unexpected directory as the
> Spamassassin line in /var/log/maillog refers to an unexpected file,
> ie. /mnt/ebs01/vmail/landplan.dk/XXX/SpamAssassin/user_prefs. By the
> way, that file does not exist.
>
> Below is my postfix configuration
>
> [ec2-user@ec2 ~]$ postconf -n
> alias_maps = hash:/etc/aliases
> broken_sasl_auth_clients = yes
> config_directory = /etc/postfix
> debug_peer_list = 85.191.189.106
> disable_vrfy_command = yes
> message_size_limit = 20480000
> myhostname = ec2.netmaster.dk
> recipient_bcc_maps = hash:/etc/postfix/recipient_bcc
> smtp_tls_loglevel = 1
> smtp_use_tls = yes
> smtpd_client_restrictions =
> smtpd_helo_required = yes
> smtpd_helo_restrictions =
> smtpd_recipient_restrictions = permit_mynetworks,
> permit_sasl_authenticated,    reject_unauth_destination,
> reject_invalid_hostname,    reject_unauth_pipelining,
> reject_non_fqdn_sender,    reject_unknown_sender_domain,
> reject_non_fqdn_recipient,    reject_unknown_recipient_domain, permit
> smtpd_sasl_auth_enable = yes
> smtpd_sasl_authenticated_header = yes
> smtpd_sasl_local_domain =
> smtpd_sasl_path = private/auth
> smtpd_sasl_security_options = noanonymous
> smtpd_sasl_type = dovecot
> smtpd_sender_restrictions =
> smtpd_tls_auth_only = yes
> smtpd_tls_cert_file =
> /etc/letsencrypt/live/mail.netmaster.dk/fullchain.pem
> smtpd_tls_key_file = /etc/letsencrypt/live/mail.netmaster.dk/privkey.pem
> smtpd_tls_loglevel = 1
> smtpd_use_tls = yes
> strict_rfc821_envelopes = yes
> virtual_alias_maps = mysql:/etc/postfix/mysql/virtual_alias_maps.cf
> virtual_gid_maps = static:2000
> virtual_mailbox_base = /mnt/ebs01/vmail
> virtual_mailbox_domains =
> mysql:/etc/postfix/mysql/virtual_domains_maps.cf
> virtual_mailbox_maps = mysql:/etc/postfix/mysql/virtual_mailbox_maps.cf
> virtual_minimum_uid = 2000
> virtual_transport = lmtp:unix:private/dovecot-lmtp
> virtual_uid_maps = static:2000
> [ec2-user@ec2 ~]$
>
> Any help would be appreciated
>
> Thomas S
>
You deliver mail to dovecot, so dovecot is responsible for writing to
the mail store. Could it be dovecot that is creating the unexpected
directories?

John

Reply | Threaded
Open this post in threaded view
|

Re: Unexpected directories in virtual_mailbox_base

Thomas Seilund

On 27/02/2019 23.50, John Fawcett wrote:

> On 27/02/2019 17:56, Thomas Seilund wrote:
>> Hi All,
>>
>> I run a mail server with Postfix (version 2.6.6), Dovecot and
>> Spamassassin.
>>
>> The first time I saw an unexpected directory in virtual_mailbox_base
>> what medio dec. 2018. The mail server has been running for 5+ years.
>>
>> There are more directories than the six directories I expect. I expect
>> one directory for each of the domains that the mail server handles.
> What were the directories that you did not expect?

/mnt/ebs01/vmail/landplan.dk is one if the directories that I do not expect.

Because landsplan.dk is not a local domain on the the mail server

 

>
>> Looking at the /var/log/maillog it seems as if:
>>
>> 1. An user that is handled by the mail server logs in and sends a mail
>> to an address that is not handled by the mail server
>>
>> 2. The mail is queued
>>
>> 3. Spamassassin kicks in and scans the mail. I don't understand that
>> as the mail is outgoing.
> You can scan outgoing mail if you want to make sure your users don't
> send viruses. Even though I trust all my users I do it as a precaution
> in case some users clients get infected and start sending viruses.
>> Perhaps Spamassassin creates the unexpected directory as the
>> Spamassassin line in /var/log/maillog refers to an unexpected file,
>> ie. /mnt/ebs01/vmail/landplan.dk/XXX/SpamAssassin/user_prefs. By the
>> way, that file does not exist.
>>
>> Below is my postfix configuration
>>
>> [ec2-user@ec2 ~]$ postconf -n
>> alias_maps = hash:/etc/aliases
>> broken_sasl_auth_clients = yes
>> config_directory = /etc/postfix
>> debug_peer_list = 85.191.189.106
>> disable_vrfy_command = yes
>> message_size_limit = 20480000
>> myhostname = ec2.netmaster.dk
>> recipient_bcc_maps = hash:/etc/postfix/recipient_bcc
>> smtp_tls_loglevel = 1
>> smtp_use_tls = yes
>> smtpd_client_restrictions =
>> smtpd_helo_required = yes
>> smtpd_helo_restrictions =
>> smtpd_recipient_restrictions = permit_mynetworks,
>> permit_sasl_authenticated,    reject_unauth_destination,
>> reject_invalid_hostname,    reject_unauth_pipelining,
>> reject_non_fqdn_sender,    reject_unknown_sender_domain,
>> reject_non_fqdn_recipient,    reject_unknown_recipient_domain, permit
>> smtpd_sasl_auth_enable = yes
>> smtpd_sasl_authenticated_header = yes
>> smtpd_sasl_local_domain =
>> smtpd_sasl_path = private/auth
>> smtpd_sasl_security_options = noanonymous
>> smtpd_sasl_type = dovecot
>> smtpd_sender_restrictions =
>> smtpd_tls_auth_only = yes
>> smtpd_tls_cert_file =
>> /etc/letsencrypt/live/mail.netmaster.dk/fullchain.pem
>> smtpd_tls_key_file = /etc/letsencrypt/live/mail.netmaster.dk/privkey.pem
>> smtpd_tls_loglevel = 1
>> smtpd_use_tls = yes
>> strict_rfc821_envelopes = yes
>> virtual_alias_maps = mysql:/etc/postfix/mysql/virtual_alias_maps.cf
>> virtual_gid_maps = static:2000
>> virtual_mailbox_base = /mnt/ebs01/vmail
>> virtual_mailbox_domains =
>> mysql:/etc/postfix/mysql/virtual_domains_maps.cf
>> virtual_mailbox_maps = mysql:/etc/postfix/mysql/virtual_mailbox_maps.cf
>> virtual_minimum_uid = 2000
>> virtual_transport = lmtp:unix:private/dovecot-lmtp
>> virtual_uid_maps = static:2000
>> [ec2-user@ec2 ~]$
>>
>> Any help would be appreciated
>>
>> Thomas S
>>
> You deliver mail to dovecot, so dovecot is responsible for writing to
> the mail store. Could it be dovecot that is creating the unexpected
> directories?
>
> John
>
Reply | Threaded
Open this post in threaded view
|

Re: Unexpected directories in virtual_mailbox_base

Bill Cole-3
On 28 Feb 2019, at 0:55, Thomas Seilund wrote:

> On 27/02/2019 23.50, John Fawcett wrote:
>> On 27/02/2019 17:56, Thomas Seilund wrote:
>>> Hi All,
>>>
>>> I run a mail server with Postfix (version 2.6.6), Dovecot and
>>> Spamassassin.
>>>
>>> The first time I saw an unexpected directory in virtual_mailbox_base
>>> what medio dec. 2018. The mail server has been running for 5+ years.
>>>
>>> There are more directories than the six directories I expect. I
>>> expect
>>> one directory for each of the domains that the mail server handles.
>> What were the directories that you did not expect?
>
> /mnt/ebs01/vmail/landplan.dk is one if the directories that I do not
> expect.
>
> Because landsplan.dk is not a local domain on the the mail server

I'm guessing: is it the domain of the intended (non-local) recipient?

>>> Looking at the /var/log/maillog it seems as if:
>>>
>>> 1. An user that is handled by the mail server logs in and sends a
>>> mail
>>> to an address that is not handled by the mail server
>>>
>>> 2. The mail is queued
>>>
>>> 3. Spamassassin kicks in and scans the mail. I don't understand that
>>> as the mail is outgoing.
>> You can scan outgoing mail if you want to make sure your users don't
>> send viruses. Even though I trust all my users I do it as a
>> precaution
>> in case some users clients get infected and start sending viruses.
>>> Perhaps Spamassassin creates the unexpected directory as the
>>> Spamassassin line in /var/log/maillog refers to an unexpected file,
>>> ie. /mnt/ebs01/vmail/landplan.dk/XXX/SpamAssassin/user_prefs. By the
>>> way, that file does not exist.

Guessing: XXX is the local part of the intended recipient?

Depending on its configuration, SpamAssassin may create a per-user
configuration directory (by default, ~/.spamassassin/, but it can be set
to ~/SpamAssassin/ or anything else you choose) for whatever user for
whom it believes it is scanning a message. For virtual users, the
expansion of "~" is configurable as well and typically looks like

What I THINK is happening is that you have hooked into SpamAssassin in
such a way that it is being told unconditionally that it should use the
per-user preferences of the recipient of the message.  I think your goal
should be to either make SA scan outbound without per-user
configurations or to  not scan outbound at all. Because the plumbing
between Postfix and SA can vary greatly (standalone milter, MIMEDefang,
Amavis as a milter, Amavis as a SMTP proxy, content_filter, pipe to
spamc...) I won't try to guess what your specific fix is. How have you
configured Postfix to pass messages to SpamAssassin?

It might be useful to also ask this question on the SpamAssassin-Users
mailing list, as the actual directory creation is almost certainly being
done by SA. However, how exactly you fix it is very much a Postfix
question because Postfix is giving SA the recipient address as a user
who needs a preferences directory of their own.

--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Reply | Threaded
Open this post in threaded view
|

Re: Unexpected directories in virtual_mailbox_base

Thomas Seilund

On 28/02/2019 22.38, Bill Cole wrote:

> On 28 Feb 2019, at 0:55, Thomas Seilund wrote:
>
>> On 27/02/2019 23.50, John Fawcett wrote:
>>> On 27/02/2019 17:56, Thomas Seilund wrote:
>>>> Hi All,
>>>>
>>>> I run a mail server with Postfix (version 2.6.6), Dovecot and
>>>> Spamassassin.
>>>>
>>>> The first time I saw an unexpected directory in virtual_mailbox_base
>>>> what medio dec. 2018. The mail server has been running for 5+ years.
>>>>
>>>> There are more directories than the six directories I expect. I expect
>>>> one directory for each of the domains that the mail server handles.
>>> What were the directories that you did not expect?
>>
>> /mnt/ebs01/vmail/landplan.dk is one if the directories that I do not
>> expect.
>>
>> Because landsplan.dk is not a local domain on the the mail server
>
> I'm guessing: is it the domain of the intended (non-local) recipient?
Yes, you are right

>
>>>> Looking at the /var/log/maillog it seems as if:
>>>>
>>>> 1. An user that is handled by the mail server logs in and sends a mail
>>>> to an address that is not handled by the mail server
>>>>
>>>> 2. The mail is queued
>>>>
>>>> 3. Spamassassin kicks in and scans the mail. I don't understand that
>>>> as the mail is outgoing.
>>> You can scan outgoing mail if you want to make sure your users don't
>>> send viruses. Even though I trust all my users I do it as a precaution
>>> in case some users clients get infected and start sending viruses.
>>>> Perhaps Spamassassin creates the unexpected directory as the
>>>> Spamassassin line in /var/log/maillog refers to an unexpected file,
>>>> ie. /mnt/ebs01/vmail/landplan.dk/XXX/SpamAssassin/user_prefs. By the
>>>> way, that file does not exist.
>
> Guessing: XXX is the local part of the intended recipient?
Yes, you are right

>
> Depending on its configuration, SpamAssassin may create a per-user
> configuration directory (by default, ~/.spamassassin/, but it can be
> set to ~/SpamAssassin/ or anything else you choose) for whatever user
> for whom it believes it is scanning a message. For virtual users, the
> expansion of "~" is configurable as well and typically looks like
>
> What I THINK is happening is that you have hooked into SpamAssassin in
> such a way that it is being told unconditionally that it should use
> the per-user preferences of the recipient of the message.  I think
> your goal should be to either make SA scan outbound without per-user
> configurations or to  not scan outbound at all. Because the plumbing
> between Postfix and SA can vary greatly (standalone milter,
> MIMEDefang, Amavis as a milter, Amavis as a SMTP proxy,
> content_filter, pipe to spamc...) I won't try to guess what your
> specific fix is. How have you configured Postfix to pass messages to
> SpamAssassin?
>
> It might be useful to also ask this question on the SpamAssassin-Users
> mailing list, as the actual directory creation is almost certainly
> being done by SA. However, how exactly you fix it is very much a
> Postfix question because Postfix is giving SA the recipient address as
> a user who needs a preferences directory of their own.

It was never my intention to let SA scan outgoing messages.

I followed this guide when I set up SA -
https://wiki.apache.org/spamassassin/IntegratedSpamdInPostfix

I have added these two lines to postfix master.cf:

smtp      inet  n       -       n       -       -       smtpd -o
content_filter=spamfilter -o receive_override_options=no_address_mappings
spamfilter    unix  -       n       n       -       -       pipe
flags=Rq user=vmail argv=/usr/bin/spamfilter.sh -oi -f ${sender}
${recipient}

Furthermore, I have this script in /usr/bin/spamfilter:

#!/bin/bash
SENDMAIL=/usr/sbin/sendmail
SPAMASSASSIN=/usr/bin/spamc
RECEIVER=`echo $4 | tr '[:upper:]' '[:lower:]'`
${SPAMASSASSIN} -u $RECEIVER | ${SENDMAIL} "$@"
exit $?

Finally, this is the parameters I have for SA in file
/etc/sysconfig/spamassassin:

SPAMDOPTIONS="--daemonize --create-prefs --max-children=5
--helper-home-dir=/mnt/ebs01/vmail/%d/%l/SpamAssassin --username=vmail
--nouser-config --virtual-config-dir=/mnt/ebs01/vmail/%d/%l/SpamAssassin"
export PYTHONPATH=/usr/lib/python2.6/site-packages

>
> --
> Bill Cole
> [hidden email] or [hidden email]
> (AKA @grumpybozo and many *@billmail.scconsult.com addresses)
> Available For Hire: https://linkedin.com/in/billcole
Reply | Threaded
Open this post in threaded view
|

Re: Unexpected directories in virtual_mailbox_base

Andrey Repin-2
Greetings, Thomas Seilund!


> smtp      inet  n       -       n       -       -       smtpd -o
> content_filter=spamfilter -o receive_override_options=no_address_mappings
> spamfilter    unix  -       n       n       -       -       pipe
> flags=Rq user=vmail argv=/usr/bin/spamfilter.sh -oi -f ${sender}
> ${recipient}

> Furthermore, I have this script in /usr/bin/spamfilter:

> #!/bin/bash
> SENDMAIL=/usr/sbin/sendmail
> SPAMASSASSIN=/usr/bin/spamc
> RECEIVER=`echo $4 | tr '[:upper:]' '[:lower:]'`
> ${SPAMASSASSIN} -u $RECEIVER | ${SENDMAIL} "$@"
> exit $?

> Finally, this is the parameters I have for SA in file
> /etc/sysconfig/spamassassin:

> SPAMDOPTIONS="--daemonize --create-prefs --max-children=5
> --helper-home-dir=/mnt/ebs01/vmail/%d/%l/SpamAssassin --username=vmail
> --nouser-config --virtual-config-dir=/mnt/ebs01/vmail/%d/%l/SpamAssassin"
> export PYTHONPATH=/usr/lib/python2.6/site-packages

If your users are not using personal spamassasin lists, you can just tell it
to use same user for all server works.


--
With best regards,
Andrey Repin
Friday, March 1, 2019 10:37:52

Sorry for my terrible english...
Reply | Threaded
Open this post in threaded view
|

Re: Unexpected directories in virtual_mailbox_base

Thomas Seilund

On 01/03/2019 08.39, Andrey Repin wrote:

> Greetings, Thomas Seilund!
>
>
>> smtp      inet  n       -       n       -       -       smtpd -o
>> content_filter=spamfilter -o receive_override_options=no_address_mappings
>> spamfilter    unix  -       n       n       -       -       pipe
>> flags=Rq user=vmail argv=/usr/bin/spamfilter.sh -oi -f ${sender}
>> ${recipient}
>> Furthermore, I have this script in /usr/bin/spamfilter:
>> #!/bin/bash
>> SENDMAIL=/usr/sbin/sendmail
>> SPAMASSASSIN=/usr/bin/spamc
>> RECEIVER=`echo $4 | tr '[:upper:]' '[:lower:]'`
>> ${SPAMASSASSIN} -u $RECEIVER | ${SENDMAIL} "$@"
>> exit $?
>> Finally, this is the parameters I have for SA in file
>> /etc/sysconfig/spamassassin:
>> SPAMDOPTIONS="--daemonize --create-prefs --max-children=5
>> --helper-home-dir=/mnt/ebs01/vmail/%d/%l/SpamAssassin --username=vmail
>> --nouser-config --virtual-config-dir=/mnt/ebs01/vmail/%d/%l/SpamAssassin"
>> export PYTHONPATH=/usr/lib/python2.6/site-packages
> If your users are not using personal spamassasin lists, you can just tell it
> to use same user for all server works.

I assume I do use personal SA lists as I run like this:

-- Each user has a LearnAsSpam and LearnAsHam mailfolder.

-- I instruct users to move mails that SA falsely did not tag as spam to
the LearnAsSpam folder

-- I instruct users to have at least 10 not spam messages in LearnAsHam

-- Once a day for each user I clear the bayes files and rebuild bayes
files with:

-- sudo -u vmail sa-learn --username vmail --spam --dbpath
$SUBDIR/SpamAssassin $SUBDIR/mail/LearnAsSpam/cur

-- sudo -u vmail sa-learn --username vmail --ham  --dbpath
$SUBDIR/SpamAssassin $SUBDIR/mail/LearnAsHam/cur

-- $SUBDIR evaluates to each users vmail directory, ie.
/mnt/ebs01/vmail/netmaster.dk/tps

If there is a better way to keep bayes upto date I would be happy to know.

BTW, thanks for all the help

>
>
Reply | Threaded
Open this post in threaded view
|

Re: Unexpected directories in virtual_mailbox_base

LuKreme
On 01 Mar 2019, at 07:21, Thomas Seilund <[hidden email]> wrote:
> -- Once a day for each user I clear the bayes files and rebuild bayes files with:

You are removing the bases entries daily and rebuilding them based on a very few (if any) messages in your LaernAs folders?

That’s the same as not using bayes at all.

--
"...Life is not a journey to the grave with the intention of arriving
safely in one pretty and well-preserved piece, but to slide across the
finish line broadside, thoroughly used up, worn out, leaking oil, and
shouting GERONIMO!!!" -- Bill McKenna

Reply | Threaded
Open this post in threaded view
|

Re: Unexpected directories in virtual_mailbox_base

Thomas Seilund


On 02/03/2019 13.38, @lbutlr wrote:
On 01 Mar 2019, at 07:21, Thomas Seilund [hidden email] wrote:
-- Once a day for each user I clear the bayes files and rebuild bayes files with:
You are removing the bases entries daily and rebuilding them based on a very few (if any) messages in your LaernAs folders?

That’s the same as not using bayes at all.

Thanks for your reply

Each user has a ham mail folder and a spam mail folder.

I instruct user to have at least 10 not spam mails in the ham folder.

And I instruct the users to move spam that make it to the inbox to the spam folder.

In most cases users have +10 mails in the ham folder and +100 mails in the spam folder.

How should SA learn from the two folders if not by running sa-learn on each of the two folders regularly?

I use SA by integrating SpamAssassin into Postfix using spamd and I wrote a bash script to rewrite spam method

as described in https://wiki.apache.org/spamassassin/IntegratedSpamdInPostfix

Any advice would be wellcome


Reply | Threaded
Open this post in threaded view
|

Re: Unexpected directories in virtual_mailbox_base

Matus UHLAR - fantomas
>>On 01 Mar 2019, at 07:21, Thomas Seilund <[hidden email]> wrote:
>>>-- Once a day for each user I clear the bayes files and rebuild bayes files with:

>On 02/03/2019 13.38, @lbutlr wrote:
>>You are removing the bases entries daily and rebuilding them based on a very few (if any) messages in your LaernAs folders?
>>
>>That’s the same as not using bayes at all.

On 03.03.19 11:27, Thomas Seilund wrote:
>Each user has a ham mail folder and a spam mail folder.
>
>I instruct user to have at least 10 not spam mails in the ham folder.

spamassassin needs at least 100  pieces of each to start hitting.

>And I instruct the users to move spam that make it to the inbox to the
>spam folder.
>
>In most cases users have +10 mails in the ham folder and +100 mails in
>the spam folder.
>
>How should SA learn from the two folders if not by running sa-learn on
>each of the two folders regularly?

note that the commands you have mentioned:
https://marc.info/?l=postfix-users&m=155145015801188&w=2

don't clear bayes database, they only train new spam/ham.

the complaint was about removing bayes database daily. You don't need to
remove bayes database at all. Don't clear the bayes database.

>I use SA by integrating SpamAssassin into Postfix using spamd and I
>wrote a bash script to rewrite spam method
>
>as described in
>https://wiki.apache.org/spamassassin/IntegratedSpamdInPostfix

there are better way to integrate spamassassin to postfix, I'd recommend
spamass-milter if you weant to keep per-user databases.

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Honk if you love peace and quiet.
Reply | Threaded
Open this post in threaded view
|

Re: Unexpected directories in virtual_mailbox_base

Bill Cole-3
In reply to this post by Thomas Seilund
On 1 Mar 2019, at 9:21, Thomas Seilund wrote:

> On 01/03/2019 08.39, Andrey Repin wrote:
>> Greetings, Thomas Seilund!
>>
>>
>>> smtp      inet  n       -       n      
>>> -       -       smtpd -o
>>> content_filter=spamfilter -o
>>> receive_override_options=no_address_mappings
>>> spamfilter    unix  -       n       n      
>>> -       -       pipe
>>> flags=Rq user=vmail argv=/usr/bin/spamfilter.sh -oi -f ${sender}
>>> ${recipient}

Apparently, the reason you're filtering outbound mail is that you are
having local users submit mail on port 25, using the same configuration
of the smtpd daemon that is used for mail coming in from the Internet.

Best practice is to have port 587 "submission" (plaintext with STARTTLS
support) and/or port 465 "smtps" ("wrappermode" TLS) transports, using
smtpd with settings suited only for initial message submission. By
splitting initial message submission from inbound message transport, you
can make both services better and safer. This includes the options to
not scan mail from your own users OR to scan it differently so that you
don't create useless and unwanted directories for random remote
recipients.

>>> Furthermore, I have this script in /usr/bin/spamfilter:
>>> #!/bin/bash
>>> SENDMAIL=/usr/sbin/sendmail
>>> SPAMASSASSIN=/usr/bin/spamc
>>> RECEIVER=`echo $4 | tr '[:upper:]' '[:lower:]'`
>>> ${SPAMASSASSIN} -u $RECEIVER | ${SENDMAIL} "$@"
>>> exit $?

That's almost the simplest shim possible between Postfix and
SpamAssassin. To make it not try to use per-user configurations, just
remove the "-u $RECEIVER" on the 5th line. That would be an appropriate
script for use as the pipe target of an additional transport used as the
content_filter of a submission or smtps service.

>>> Finally, this is the parameters I have for SA in file
>>> /etc/sysconfig/spamassassin:
>>> SPAMDOPTIONS="--daemonize --create-prefs --max-children=5
>>> --helper-home-dir=/mnt/ebs01/vmail/%d/%l/SpamAssassin
>>> --username=vmail
>>> --nouser-config
>>> --virtual-config-dir=/mnt/ebs01/vmail/%d/%l/SpamAssassin"
>>> export PYTHONPATH=/usr/lib/python2.6/site-packages

Easiest way to stop creating the unwanted directories: remove
"--create-prefs" there. It won't solve the root cause, but it will fix
the symptom.

>> If your users are not using personal spamassasin lists, you can just
>> tell it
>> to use same user for all server works.
>
> I assume I do use personal SA lists as I run like this:
>
> -- Each user has a LearnAsSpam and LearnAsHam mailfolder.
>
> -- I instruct users to move mails that SA falsely did not tag as spam
> to the LearnAsSpam folder
>
> -- I instruct users to have at least 10 not spam messages in
> LearnAsHam
>
> -- Once a day for each user I clear the bayes files and rebuild bayes
> files with:
>
> -- sudo -u vmail sa-learn --username vmail --spam --dbpath
> $SUBDIR/SpamAssassin $SUBDIR/mail/LearnAsSpam/cur
>
> -- sudo -u vmail sa-learn --username vmail --ham  --dbpath
> $SUBDIR/SpamAssassin $SUBDIR/mail/LearnAsHam/cur
>
> -- $SUBDIR evaluates to each users vmail directory, ie.
> /mnt/ebs01/vmail/netmaster.dk/tps
>
> If there is a better way to keep bayes upto date I would be happy to
> know.

Your users are unlikely to be actually using Bayes if you're clearing
the databases daily. SA Bayes will not score messages AT ALL if its
database doesn't have enough messages learned to have a statistically
valid sample size, set by default to 200 each of spam and ham. That's
high enough to avoid most cases of Bayes being actively bad, but Bayes
doesn't really work *well* until it has about a thousand messages
analyzed.

--
Bill Cole
[hidden email] or [hidden email]
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole
Reply | Threaded
Open this post in threaded view
|

Re: Unexpected directories in virtual_mailbox_base

Thomas Seilund

On 03/03/2019 21.31, Bill Cole wrote:

> On 1 Mar 2019, at 9:21, Thomas Seilund wrote:
>
>> On 01/03/2019 08.39, Andrey Repin wrote:
>>> Greetings, Thomas Seilund!
>>>
>>>
>>>> smtp      inet  n       - n       -       -       smtpd -o
>>>> content_filter=spamfilter -o
>>>> receive_override_options=no_address_mappings
>>>> spamfilter    unix  -       n       n       -       - pipe
>>>> flags=Rq user=vmail argv=/usr/bin/spamfilter.sh -oi -f ${sender}
>>>> ${recipient}
>
> Apparently, the reason you're filtering outbound mail is that you are
> having local users submit mail on port 25, using the same
> configuration of the smtpd daemon that is used for mail coming in from
> the Internet.
Your are right. The local users are now using port 587 for outbound
mail. And I have added "-o smtpd_sasl_auth_enable=no" to smtp entry in
master.cf as my default value for smtpd_sasl_auth_enable is yes. This
way I hope that new users will not by accident set up port 25 for
outbound mail. Thanks a lot for the tip!
>
> Best practice is to have port 587 "submission" (plaintext with
> STARTTLS support) and/or port 465 "smtps" ("wrappermode" TLS)
> transports, using smtpd with settings suited only for initial message
> submission. By splitting initial message submission from inbound
> message transport, you can make both services better and safer. This
> includes the options to not scan mail from your own users OR to scan
> it differently so that you don't create useless and unwanted
> directories for random remote recipients.
It was intention to follow this best practice!

>
>>>> Furthermore, I have this script in /usr/bin/spamfilter:
>>>> #!/bin/bash
>>>> SENDMAIL=/usr/sbin/sendmail
>>>> SPAMASSASSIN=/usr/bin/spamc
>>>> RECEIVER=`echo $4 | tr '[:upper:]' '[:lower:]'`
>>>> ${SPAMASSASSIN} -u $RECEIVER | ${SENDMAIL} "$@"
>>>> exit $?
>
> That's almost the simplest shim possible between Postfix and
> SpamAssassin. To make it not try to use per-user configurations, just
> remove the "-u $RECEIVER" on the 5th line. That would be an
> appropriate script for use as the pipe target of an additional
> transport used as the content_filter of a submission or smtps service.
The reason I use per-user configuration is because I want each user to
have his or her own bayes-filter. Is that the correct way to get to that
situation?

>
>
>>>> Finally, this is the parameters I have for SA in file
>>>> /etc/sysconfig/spamassassin:
>>>> SPAMDOPTIONS="--daemonize --create-prefs --max-children=5
>>>> --helper-home-dir=/mnt/ebs01/vmail/%d/%l/SpamAssassin --username=vmail
>>>> --nouser-config
>>>> --virtual-config-dir=/mnt/ebs01/vmail/%d/%l/SpamAssassin"
>>>> export PYTHONPATH=/usr/lib/python2.6/site-packages
>
> Easiest way to stop creating the unwanted directories: remove
> "--create-prefs" there. It won't solve the root cause, but it will fix
> the symptom.
As you mentioned above the root cause is that users submit mail on port
25. That has been fixed!

>
>>> If your users are not using personal spamassasin lists, you can just
>>> tell it
>>> to use same user for all server works.
>>
>> I assume I do use personal SA lists as I run like this:
>>
>> -- Each user has a LearnAsSpam and LearnAsHam mailfolder.
>>
>> -- I instruct users to move mails that SA falsely did not tag as spam
>> to the LearnAsSpam folder
>>
>> -- I instruct users to have at least 10 not spam messages in LearnAsHam
>>
>> -- Once a day for each user I clear the bayes files and rebuild bayes
>> files with:
>>
>> -- sudo -u vmail sa-learn --username vmail --spam --dbpath
>> $SUBDIR/SpamAssassin $SUBDIR/mail/LearnAsSpam/cur
>>
>> -- sudo -u vmail sa-learn --username vmail --ham  --dbpath
>> $SUBDIR/SpamAssassin $SUBDIR/mail/LearnAsHam/cur
>>
>> -- $SUBDIR evaluates to each users vmail directory, ie.
>> /mnt/ebs01/vmail/netmaster.dk/tps
>>
>> If there is a better way to keep bayes upto date I would be happy to
>> know.
>
> Your users are unlikely to be actually using Bayes if you're clearing
> the databases daily. SA Bayes will not score messages AT ALL if its
> database doesn't have enough messages learned to have a statistically
> valid sample size, set by default to 200 each of spam and ham. That's
> high enough to avoid most cases of Bayes being actively bad, but Bayes
> doesn't really work *well* until it has about a thousand messages
> analyzed.
>
I am confused about Bayes-files. What is the best strategy to fight
spam? Is Bayes-files a good idea at all and if so how do I organize
spam-fighting using SA Bayes?