smtpd_proxy_filter disables smtpd_milters?

[Hans van Zijst]

I'm trying to do all my scanning before-queue: spf, dkim, dmarc and
SpamAssassin. There are strong arguments to do SpamAssassin after the
message has been queued, but for this one I need everything before-queue.

I've defined OpenDKIM and OpenDMARC as smtpd_milters, using UNIX
sockets. In the "normal" setup they work like a charm, but now I add
Amavis to the mix, with smtpd_proxy_filter, and suddenly both milters
disappear. Why?

This is the relevant part of my master.cf:

smtp      inet  n       -       y       -       1       postscreen

smtpd    pass  -       -       y       -       -      smtpd
        -o syslog_name=postfix/smtpd_proxy_filter
        -o smtpd_client_connection_count_limit=10
        -o receive_override_options=
        -o smtpd_proxy_filter=127.0.0.1:10024

127.0.0.1:10025 inet n  -       n       -        -      smtpd
        -o syslog_name=postfix/smtpd_10025
        -o smtpd_tls_security_level=none
        -o smtpd_authorized_xforward_hosts=127.0.0.0/8
        -o smtpd_client_restrictions=
        -o smtpd_helo_restrictions=
        -o smtpd_sender_restrictions=
        -o smtpd_relay_restrictions=
        -o smtpd_recipient_restrictions=permit_mynetworks,reject
        -o smtpd_data_restrictions=
        -o mynetworks=127.0.0.0/8
        -o
receive_override_options=no_unknown_recipient_checks,no_header_body_checks,no_unknown_recipient_checks,no_milters


The milters are defined in main.cf as:

milter_protocol = 6
milter_default_action = accept
smtpd_milters = unix:private/opendkim,unix:private/opendmarc


To my understanding, they should be called before smtpd hands the
message to the filter on 127.0.0.1:10024, which is Amavis. If I turn on
debugging, it seems like they actually do get called:


Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]:
milter8_connect: non-protocol events for protocol version 6:
Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]:
milter8_connect: transport=unix endpoint=private/opendkim
Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]:
milter8_connect: my_version=0x6
Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]:
milter8_connect: my_actions=0x1ff SMFIF_ADDHDRS SMFIF_CHGBODY
SMFIF_ADDRCPT SMFIF_DELRCPT SMFIF_CHGHDRS SMFIF_QUARANTINE SMFIF_CHGFROM
SMFIF_ADDRCPT_PAR SMFIF_SETSYMLIST
Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]:
milter8_connect: my_events=0x1fffff SMFIP_NOCONNECT SMFIP_NOHELO
SMFIP_NOMAIL SMFIP_NORCPT SMFIP_NOBODY SMFIP_NOHDRS SMFIP_NOEOH
SMFIP_NR_HDR SMFIP_NOUNKNOWN SMFIP_NODATA SMFIP_SKIP SMFIP_RCPT_REJ
SMFIP_NR_CONN SMFIP_NR_HELO SMFIP_NR_MAIL SMFIP_NR_RCPT SMFIP_NR_DATA
SMFIP_NR_UNKN SMFIP_NR_EOH SMFIP_NR_BODY SMFIP_HDR_LEADSPC
Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]:
milter8_connect: milter unix:private/opendkim version 6
Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]:
milter8_connect: events SMFIP_NOHELO SMFIP_NOUNKNOWN SMFIP_NODATA
SMFIP_SKIP SMFIP_HDR_LEADSPC
Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]:
milter8_connect: requests SMFIF_ADDHDRS SMFIF_CHGHDRS SMFIF_SETSYMLIST
Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]:
milter_macro_lookup: "j"
Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]:
milter_macro_lookup: result "<my mailhost>"
Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]:
milter_macro_lookup: "{daemon_name}"
Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]:
milter_macro_lookup: result "<my mailhost>"
Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]:
milter_macro_lookup: "{daemon_addr}"
Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]:
milter_macro_lookup: result "<my IP>"
Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]:
milter_macro_lookup: "v"
Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]:
milter_macro_lookup: result "Postfix 3.4.14"
Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]:
milter8_conn_event: milter unix:private/opendkim: connect <client>
Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]: event:
SMFIC_CONNECT; macros: j=<my mailhost> {daemon_name}=<my mailhost>
{daemon_addr}=<my IP> v=Postfix 3.4.14
Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]: reply:
SMFIR_CONTINUE data 0 bytes
Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]:
milter8_connect: non-protocol events for protocol version 6:
Sep 21 16:55:56 host971 postfix/smtpd_proxy_filter/smtpd[39984]:
milter8_connect: transport=unix endpoint=private/opendmarc


But neither OpenDKIM nor OpenDMARC adds a header and if I turn off
debugging, there's no mention of either of them in the log.

What am I missing here? Is my assumption that the milters should called
in smtpd correct, or does the use of smtpd_proxy_filter imply something
that I don't know of?

Kind regards,

Hans

[Damian Lukowski]

> I've defined OpenDKIM and OpenDMARC as smtpd_milters, using UNIX
> sockets. In the "normal" setup they work like a charm, but now I add
> Amavis to the mix, with smtpd_proxy_filter, and suddenly both milters
> disappear. Why?

From [1]:

> When you use the before-queue content filter for incoming SMTP mail
> (see SMTPD_PROXY_README), Milter applications have access only to the
> SMTP command information; they have no access to the message header or
> body, and cannot make modifications to the message or to the envelope.
You can use amavisd-milter, though.

[1] http://www.postfix.org/MILTER_README.html

[Hans van Zijst]

Hi Damian,

Thanks for your quick answer.

On 21-09-2020 18:03, Damian wrote:

>> I've defined OpenDKIM and OpenDMARC as smtpd_milters, using UNIX
>> sockets. In the "normal" setup they work like a charm, but now I add
>> Amavis to the mix, with smtpd_proxy_filter, and suddenly both milters
>> disappear. Why?
>
> From [1]:
>
>> When you use the before-queue content filter for incoming SMTP mail
>> (see SMTPD_PROXY_README), Milter applications have access only to the
>> SMTP command information; they have no access to the message header or
>> body, and cannot make modifications to the message or to the envelope.
> You can use amavisd-milter, though.
>
> [1] http://www.postfix.org/MILTER_README.html

Can't understand that I really missed that part...

I'll fix my setup with amavisd-milter, which makes the setup simpler as
well.

Kind regards,

Hans

[Jaroslaw Rafa]

Dnia 21.09.2020 o godz. 18:15:00 Hans van Zijst pisze:
> I'll fix my setup with amavisd-milter, which makes the setup simpler as
> well.

Is there anything *besides* running SpamAssassin that you need Amavis for?

Because if you want *only* to run SpamAssassin, it's simpler to use
spamass-milter (or another spamassassin milter; there are 2 or 3 of them)
instead of all the complicated mechanics of Amavis.
--
Pozdrowienia,
   Jaroslaw Rafa
   [hidden email]
--
"In a million years, when kids go to school, they're gonna know: once there
was a Hushpuppy, and she lived with her daddy in the Bathtub."

[Hans van Zijst]

Hi Dnia,

On 21-09-2020 18:39, Jaroslaw Rafa wrote:
> Dnia 21.09.2020 o godz. 18:15:00 Hans van Zijst pisze:
>> I'll fix my setup with amavisd-milter, which makes the setup simpler as
>> well.
>
> Is there anything *besides* running SpamAssassin that you need Amavis for?
>
> Because if you want *only* to run SpamAssassin, it's simpler to use
> spamass-milter (or another spamassassin milter; there are 2 or 3 of them)
> instead of all the complicated mechanics of Amavis.

Yes, I also use Amavis with ClamAV for virusscanning. It should be more
efficient to let Amavis schedule the whole scanning/filtering process,
and it offers the option to let users configure their own individual
settings for it in LDAP.

But I agree that Amavis is a complex beast...

Kind regards,

Hans