Rspamd as milter and 'discard' action

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Rspamd as milter and 'discard' action

Ralph Seichter-2
I'm having trouble with Rspamd as a milter for Postfix, specifically
with Rspamd's discard action:

  # /etc/rspamd/local.d/force_actions.conf
  rules {
    FOO_RULE {
      expression = 'FOO_EXPR',
      action = 'discard'
    }
  }

If I use the 'reject' action in Rspamd rules like the one shown above,
Postfix rejects matching messages on arrival, as is expected. However,
the actions 'discard' and 'quarantine' have no visible effect.

Since Rspamd merely offers a suggestion on how the MTA is supposed to
treat messages, I probably need to configure Postfix to honor 'discard'
suggestions? I have searched for quite a while but could not find a
solution, so I am asking for advice here. Thanks.

-Ralph
Reply | Threaded
Open this post in threaded view
|

Re: Rspamd as milter and 'discard' action

Viktor Dukhovni
On Mon, Mar 11, 2019 at 09:28:40PM +0100, Ralph Seichter wrote:

> I'm having trouble with Rspamd as a milter for Postfix, specifically
> with Rspamd's discard action:
>
>   # /etc/rspamd/local.d/force_actions.conf
>   rules {
>     FOO_RULE {
>       expression = 'FOO_EXPR',
>       action = 'discard'
>     }
>   }

When do you trigger than rule?  Postfix supports 'discard', but only
after "MAIL FROM", not after CONNECT or EHLO.

            /*
             * Decision: accept and silently discard this message. According
             * to the milter API documentation there will be no action when
             * this is requested by a connection-level function. This
             * decision is final (i.e. Sendmail 8 changes receiver state).
             */
        case SMFIR_DISCARD:
            if (data_size != 0)
                break;
            if (IN_CONNECT_EVENT(event)) {
                msg_warn("milter %s: DISCARD action is not allowed "
                         "for connect or helo", milter->m.name);
                MILTER8_EVENT_BREAK(milter->def_reply);
            } else {
                /* No more events for this message. */
                milter->state = MILTER8_STAT_ACCEPT_MSG;
                MILTER8_EVENT_BREAK("D");
            }

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Rspamd as milter and 'discard' action

Ralph Seichter-2
* Viktor Dukhovni:

> When do you trigger than rule? Postfix supports 'discard', but only
> after "MAIL FROM", not after CONNECT or EHLO.

$ postconf -n | grep milter
milter_default_action = accept
non_smtpd_milters = unix:/run/opendkim/socket
smtpd_milters = unix:/run/opendkim/socket inet:localhost:11332

Is this the right way to do it?  Rspamd is listening on localhost:11332
(little surprise there). I grep'd my Postfix log file because I was
looking for clues, but 'DISCARD' does not appear anywhere. When I use
'reject' in Rspamd instead, Postfix logs look like this:

Mar 11 22:51:18 ra postfix/cleanup[12573]: D6FBA48C13A0: milter-reject: END-OF-MESSAGE from mail.dinamer.eu[89.163.155.223]: 5.7.1 rspamd objects; from=<[hidden email]> to=<[hidden email]> proto=ESMTP helo=<mail.dinamer.eu>

-Ralph
Reply | Threaded
Open this post in threaded view
|

Re: Rspamd as milter and 'discard' action

Viktor Dukhovni
> On Mar 11, 2019, at 6:04 PM, Ralph Seichter <[hidden email]> wrote:
>
>> When do you trigger than rule? Postfix supports 'discard', but only
>> after "MAIL FROM", not after CONNECT or EHLO.
>
> $ postconf -n | grep milter
> milter_default_action = accept
> non_smtpd_milters = unix:/run/opendkim/socket
> smtpd_milters = unix:/run/opendkim/socket inet:localhost:11332
>
> Is this the right way to do it?  Rspamd is listening on localhost:11332
> (little surprise there). I grep'd my Postfix log file because I was
> looking for clues, but 'DISCARD' does not appear anywhere. When I use
> 'reject' in Rspamd instead, Postfix logs look like this:

Under what conditions does the milter respond with "discard"?
It should not do that until before the "MAIL FROM" command.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: Rspamd as milter and 'discard' action

Ralph Seichter-2
* Viktor Dukhovni:

> Under what conditions does the milter respond with "discard"?

The trigger expression I mentioned before is tied to the "From"
header [1], like so:

  konfig['regexp']['FOO_EXPR'] = { [2]
    re = 'From=/user\\@domain\\.tld/Hi'
  }

I verified that the expression itself is correct.

-Ralph

[1] https://rspamd.com/doc/modules/regexp.html

[2] Sigh... The mailing list submission filter objects to the word
c-o-n-f-i-g. Is this really necessary? It feels like I run afoul of the
filter with every third message I write. :-(
Reply | Threaded
Open this post in threaded view
|

Re: Rspamd as milter and 'discard' action

Ralph Seichter-2
In reply to this post by Viktor Dukhovni
I have asked on the Rspamd mailing list because I wanted to be certain
that I did not forget anything on the Rspamd side, but the one answer I
received turned out to be a dud.

I'd really be glad for pointers.

-Ralph
Reply | Threaded
Open this post in threaded view
|

Re: Rspamd as milter and 'discard' action

Wietse Venema
Ralph Seichter:
> I have asked on the Rspamd mailing list because I wanted to be certain
> that I did not forget anything on the Rspamd side, but the one answer I
> received turned out to be a dud.
>
> I'd really be glad for pointers.

Here is one:
- set 'disable_mime_output_conversion = yes'.
- send test messages.
- find out what messages are modified and what messages are not.
- find out where in the path a message is being modified.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Rspamd as milter and 'discard' action

Ralph Seichter-2
* Wietse Venema:

> find out where in the path a message is being modified.

Hm. Are you perhaps confusing me with Michael Ludwig who posted about
his DKIM trouble? I am not experiencing modified messages, I am just
wondering why a Rspamd action of "reject" is passed to Postfix and
honored there, while "discard" is not. I can't (yet) figure out if the
problem exists within my Rspamd config, Postfix config, or if it is a
case of generalized PEBKAC.

-Ralph
Reply | Threaded
Open this post in threaded view
|

Re: Rspamd as milter and 'discard' action

Wietse Venema
Ralph Seichter:
> * Wietse Venema:
>
> > find out where in the path a message is being modified.
>
> Hm. Are you perhaps confusing me with Michael Ludwig who posted about
> his DKIM trouble?

You posted a one-line question with zero context.

> I am not experiencing modified messages, I am just
> wondering why a Rspamd action of "reject" is passed to Postfix and
> honored there, while "discard" is not. I can't (yet) figure out if the
> problem exists within my Rspamd config, Postfix config, or if it is a
> case of generalized PEBKAC.

We already answered that the Milter protocol does not allow a
"discard" request at the connection level SMTP protocol states
i.e. they require MAIL, RCPT, DATA.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Rspamd as milter and 'discard' action

Ralph Seichter-2
* Wietse Venema:

> You posted a one-line question with zero context.

Well, I thought that the header data in "In-Reply-To" and "References"
was sufficient, but I am sorry if it was not. Seems like my penchant for
brevity got the better of me. ;-)

> We already answered that the Milter protocol does not allow a
> "discard" request at the connection level SMTP protocol states
> i.e. they require MAIL, RCPT, DATA.

I am aware. As I wrote previously in a reply to Viktor's message, the
trigger expression I use is based on the "From" header, which is
transmitted after DATA. If I am not mistaken, DISCARD should be allowed
at this point, as is REJECT?

Rest assured that I'm not trying to be obstinate.

-Ralph
Reply | Threaded
Open this post in threaded view
|

Re: Rspamd as milter and 'discard' action

Wietse Venema
Ralph Seichter:

> > We already answered that the Milter protocol does not allow a
> > "discard" request at the connection level SMTP protocol states
> > i.e. they require MAIL, RCPT, DATA.
>
> I am aware. As I wrote previously in a reply to Viktor's message, the
> trigger expression I use is based on the "From" header, which is
> transmitted after DATA. If I am not mistaken, DISCARD should be allowed
> at this point, as is REJECT?
>
> Rest assured that I'm not trying to be obstinate.

Alas, I have no time to investigate this absent more concrete
information.  You may want to configure "cleanup -v" in master.cf
to log what happens when it receives the Milter's DISCARD response.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Rspamd as milter and 'discard' action

Ralph Seichter-2
* Wietse Venema:

> You may want to configure "cleanup -v" in master.cf to log what
> happens when it receives the Milter's DISCARD response.

Thank you for that. Based on the logs I am now convinced that the
problems lie with either Rspamd or how I use Rspamd based on my
interpretation of the docs (which are not as clear as I would wish).

  postfix/cleanup[3014]: 395EA48C128B: milter-discard: END-OF-MESSAGE
  from mailout12.t-online.de[194.25.134.22]: milter triggers DISCARD
  action; [...]

So, Postfix does honor DISCARD, if the milter actually answers in this
fashion. Problem is, I only managed to get Rspamd to do so by using a
special config parameter:

  # /etc/rspamd/local.d/worker-proxy.inc
  discard_on_reject = true;

If I now use the 'reject' action, Rspamd signals 'discard' instead.
Unfortunately, this affects all rejections, which is not what I need,
but it is clear that Postfix works as desired if I can somehow make
Rspamd do what I need it to do. I hope the Rspamd mailing list will lead
to a solution.

My thanks to you and Viktor for helping me rule out Postfix as the
source of my troubles.

-Ralph