Tip: Restricting mail reception using a remote service's SPF records

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Tip: Restricting mail reception using a remote service's SPF records

Ville Walveranta
Here's an idea.. maybe it's useful for someone, so I post it here.

I'm setting up a local mail server to cache remote service's mail for
faster access on the LAN. The remote server has an up-to-date SPF
record that is updated whenever the sending IP ranges change. I want
to limit unauthenticated mail reception to the sending servers of the
remote mail system. I use the following in main.cf:

smtpd_client_restrictions =
        permit_mynetworks
        permit_sasl_authenticated
        check_client_access
cidr:$config_directory/tables/client_access_maps.cidr
        reject

To automatically keep the client_access_maps.cird up-to-date I run the
following as a cron-job.. perhaps every hour. The SPF record changes
rarely, but obviously when it does, the mail would not be accepted on
the local server without an update. On the other hand, I assume when
the sending ranges change, the SPF records are updated well in advance
so there is plenty of time for the local system to pick up the change.

The following is "concept"; I don't have the script yet ready, but
it'll be easy to write with your favorite scripting language:

-------
1. get your remote sender's current SPF record:

dig yourremotesender.com txt  > tempfile

2. parse the result in tempfile with regex:

/.*?ANSWER\sSECTION:.*?yourremotesender\.com*.\s+?\d+?\s+?IN\s+?TXT\s+?"v=spf1\s+?([^)]+?)\s+?.all"/

(dot matches newline)

3. explode the result

4. remove "ip4:" from the front of each segment if present

5. mogrify to format
<ip> OK
<ip>/cidr OK

 .. and save to a temp file.

6. compare the checksum of the temp file to the currently active
client_access_maps.cidr; if different..

7. copy the tempfile to ../postfix/tables/client_access_maps.cidr

8. execute `postfix reload'
-------

Ville
Reply | Threaded
Open this post in threaded view
|

Re: Tip: Restricting mail reception using a remote service's SPF records

Barney Desmond
> The following is "concept"; I don't have the script yet ready, but
> it'll be easy to write with your favorite scripting language:
>
> -------
> 1. get your remote sender's current SPF record:
>
> dig yourremotesender.com txt  > tempfile
>
> 2. parse the result in tempfile with regex:
>
> /.*?ANSWER\sSECTION:.*?yourremotesender\.com*.\s+?\d+?\s+?IN\s+?TXT\s+?"v=spf1\s+?([^)]+?)\s+?.all"/
<snip>

The implementation seems a little fragile, but the concept should be
workable. On dig (at least the versions I've used), you can use +short
for parseable output. Example:

dig +short example.com TXT | grep 'v=spf1' | egrep -o 'ip4:[0-9./]+' |
sed 's/^ip4://' | sed 's/$/      OK/' > tempfile

Something like that, anyway.
Reply | Threaded
Open this post in threaded view
|

Re: Tip: Restricting mail reception using a remote service's SPF records

Ville Walveranta
Thanks, I'll look into that; it'll simplify it a bit.

Anything that is parsed from text output is obviously not super solid
but for this application it'll suffice. The MX for the business
domains in question is an external service that takes care of spam
filtering, address consolidation, etc. The external service also
offers better availability than the LAN server that is not monitored
24/7 and that is connected to the internet via single T1. But as a
remote service it is not as fast as accessing emails on the LAN, and
at peak-times the Internet congestion slows it further. This is a
small office environment, and the plan is to "cache" the couple of
dozen mailboxes to the LAN mail server. The above configuration will
be used to prevent any mail from outside of the external service from
finding its way to the local mail server.

In the event the local server fails (since there is no server
redundancy), users continue to have full access (albeit slower) to
their mailboxes using the external service which is the primary
receiver anyway.

Ville
Reply | Threaded
Open this post in threaded view
|

Re: Tip: Restricting mail reception using a remote service's SPF records

Ville Walveranta
Here's the completed script (the IP/CIDR extract worked perfectly --
thanks Barney!):

---
#!/bin/sh

ORIGINAL=/usr/local/etc/postfix/tables/client_access_maps.cidr
NEW=/tmp/postfix_clients.tmp

dig +short senderdomain.net TXT | grep 'v=spf1' | egrep -o
'ip4:[0-9./]+' | sed 's/^ip4://' | sed 's/$/      OK/' > $NEW

ORIGINAL_CK=`cksum $ORIGINAL | awk '{print $1}'`
NEW_CK=`cksum $NEW | awk '{print $1}'`

if [ -s $NEW ] ; then
  if [ $ORIGINAL_CK != $NEW_CK ] ; then
    cp -f $NEW $ORIGINAL
    postfix reload > /dev/null
  fi
fi

rm $NEW

exit 0
---

It works except that the Postfix refresh message
("postfix/postfix-script: refreshing the Postfix mail system") is
displayed despite of the attempt to redirect it to /dev/null?  Any
idea how I could hide it?

Ville
Reply | Threaded
Open this post in threaded view
|

Re: Tip: Restricting mail reception using a remote service's SPF records

Gerardo Herzig
Ville Walveranta wrote:

> Here's the completed script (the IP/CIDR extract worked perfectly --
> thanks Barney!):
>
> ---
> #!/bin/sh
>
> ORIGINAL=/usr/local/etc/postfix/tables/client_access_maps.cidr
> NEW=/tmp/postfix_clients.tmp
>
> dig +short senderdomain.net TXT | grep 'v=spf1' | egrep -o
> 'ip4:[0-9./]+' | sed 's/^ip4://' | sed 's/$/      OK/' > $NEW
>
> ORIGINAL_CK=`cksum $ORIGINAL | awk '{print $1}'`
> NEW_CK=`cksum $NEW | awk '{print $1}'`
>
> if [ -s $NEW ] ; then
>   if [ $ORIGINAL_CK != $NEW_CK ] ; then
>     cp -f $NEW $ORIGINAL
>     postfix reload > /dev/null
>   fi
> fi
>
> rm $NEW
>
> exit 0
> ---
>
> It works except that the Postfix refresh message
> ("postfix/postfix-script: refreshing the Postfix mail system") is
> displayed despite of the attempt to redirect it to /dev/null?  Any
> idea how I could hide it?
>
> Ville
>
>
Try
postfix reload 2>/dev/null instead

Gerardo
Reply | Threaded
Open this post in threaded view
|

Re: Tip: Restricting mail reception using a remote service's SPF records

J.P. Trosclair
In reply to this post by Ville Walveranta
Ville Walveranta wrote:
>
> It works except that the Postfix refresh message
> ("postfix/postfix-script: refreshing the Postfix mail system") is
> displayed despite of the attempt to redirect it to /dev/null?  Any
> idea how I could hide it?

It's probably writing to stderr,

postfix reload 2>/dev/null   # stderr only
postfix reload &>/dev/null   # stdout & stderr

Though if it writes an error to the screen, you'll miss it.

J.P.

Reply | Threaded
Open this post in threaded view
|

Re: Tip: Restricting mail reception using a remote service's SPF records

Roderick A. Anderson-4
In reply to this post by Ville Walveranta
Ville Walveranta wrote:

> Here's the completed script (the IP/CIDR extract worked perfectly --
> thanks Barney!):
>
> ---
> #!/bin/sh
>
> ORIGINAL=/usr/local/etc/postfix/tables/client_access_maps.cidr
> NEW=/tmp/postfix_clients.tmp
>
> dig +short senderdomain.net TXT | grep 'v=spf1' | egrep -o
> 'ip4:[0-9./]+' | sed 's/^ip4://' | sed 's/$/      OK/' > $NEW
>
> ORIGINAL_CK=`cksum $ORIGINAL | awk '{print $1}'`
> NEW_CK=`cksum $NEW | awk '{print $1}'`
>
> if [ -s $NEW ] ; then
>   if [ $ORIGINAL_CK != $NEW_CK ] ; then
>     cp -f $NEW $ORIGINAL
>     postfix reload > /dev/null
>   fi
> fi
>
> rm $NEW
>
> exit 0
> ---
>
> It works except that the Postfix refresh message
> ("postfix/postfix-script: refreshing the Postfix mail system") is
> displayed despite of the attempt to redirect it to /dev/null?  Any
> idea how I could hide it?

postfix reload > /dev/null 2>&1


Rod
--
Reply | Threaded
Open this post in threaded view
|

Re: Tip: Restricting mail reception using a remote service's SPF records

Ville Walveranta
Perfect! Thanks all!!
Reply | Threaded
Open this post in threaded view
|

Re: Tip: Restricting mail reception using a remote service's SPF records

@lbutlr
In reply to this post by Ville Walveranta
On 25-Jun-2009, at 14:33, Ville Walveranta wrote:
> It works except that the Postfix refresh message
> ("postfix/postfix-script: refreshing the Postfix mail system") is
> displayed despite of the attempt to redirect it to /dev/null?  Any
> idea how I could hide it?

That refresh message is output on stderror, and you are only  
redirecting stdout.

in /bin/sh I believe the way to redirect (assuming you are using  
bash's sh) is

2>&1


--
So here's us, on the raggedy edge. Don't push me. And
        I won't push you.

Reply | Threaded
Open this post in threaded view
|

Re: Tip: Restricting mail reception using a remote service's SPF records

mouss-4
In reply to this post by Ville Walveranta
Ville Walveranta a écrit :

> Here's an idea.. maybe it's useful for someone, so I post it here.
>
> I'm setting up a local mail server to cache remote service's mail for
> faster access on the LAN. The remote server has an up-to-date SPF
> record that is updated whenever the sending IP ranges change. I want
> to limit unauthenticated mail reception to the sending servers of the
> remote mail system. I use the following in main.cf:
>
> smtpd_client_restrictions =
>         permit_mynetworks
>         permit_sasl_authenticated
>         check_client_access
> cidr:$config_directory/tables/client_access_maps.cidr
>         reject
>
> To automatically keep the client_access_maps.cird up-to-date I run the
> following as a cron-job.. perhaps every hour. The SPF record changes
> rarely, but obviously when it does, the mail would not be accepted on
> the local server without an update. On the other hand, I assume when
> the sending ranges change, the SPF records are updated well in advance
> so there is plenty of time for the local system to pick up the change.
>
> The following is "concept"; I don't have the script yet ready, but
> it'll be easy to write with your favorite scripting language:
>
> -------
> 1. get your remote sender's current SPF record:
>
> dig yourremotesender.com txt  > tempfile
>
> 2. parse the result in tempfile with regex:
>
> /.*?ANSWER\sSECTION:.*?yourremotesender\.com*.\s+?\d+?\s+?IN\s+?TXT\s+?"v=spf1\s+?([^)]+?)\s+?.all"/
>
> (dot matches newline)
>
> 3. explode the result
>
> 4. remove "ip4:" from the front of each segment if present
>
> 5. mogrify to format
> <ip> OK
> <ip>/cidr OK
>
>  .. and save to a temp file.
>
> 6. compare the checksum of the temp file to the currently active
> client_access_maps.cidr; if different..
>
> 7. copy the tempfile to ../postfix/tables/client_access_maps.cidr
>
> 8. execute `postfix reload'


so you would block mail from me? if you want to jump the SPF wagon, it
is a good idea to go for "generalized SPF" (GSPF): also allow hosts that
match the sender domain (host.example.com can send mail on behalf on
*@example.com). if this is not acceptable, resolve smtp.example.com,
mail.example.com, mx.example.com, ... but you may also need things like
smtp-1.example.com, ...


all that said, if you whitelist IPs or blocks, please submit them to DNSWL.

Reply | Threaded
Open this post in threaded view
|

Re: Tip: Restricting mail reception using a remote service's SPF records

Ville Walveranta
On Fri, Jun 26, 2009 at 5:17 PM, mouss<[hidden email]> wrote:
> so you would block mail from me?

Yes, in fact, the local "cache" mail server would refuse mail from
everyone but the servers mentioned in the external service's SPF
record hence creating sort of an unprotected, IP-based "tunnel"
between the external service and the local server.

The only MX for the serviced domains is that of the external service,
so nobody but the external service's SMTP server should be looking for
the local server's SMTP port; the local server only receives a copy of
the received emails from the external service after spam filtering.

The reason for the local service's existence is to improve the access
speed to the received emails, and to function as an outbound SMTP for
the LAN so that emails with large attachments are sent "immediately"
from the users' point of view (even though large attachments can take
several minutes to send over 2.8Mb internet connection). The local
server's SMTP is open for "mynetworks" (the LAN), and also available
via port 587 with SASL AUTH/TLS so that the users may use it for
outbound mail.

The serviced domains' SPF records include both the external service's
SPF record (via include:) and the IP of the local server (via IP4:).
I'm not totally sold on SPF for it's spam-preventing capability, but
it is handy in automating the check_client_access CIDR map as has been
discussed in this thread. And its use possibly discourages spammers
from using the business domains by making it more likely for the
emails with forged headers to end up trapped by the spam filters.

I've tested this now intially, and it seems to be working well. It's a
bit of a kludge, but gives best of both worlds to the users at a very
low cost: stability and features of a managed external service, and
speed of a local service.

> all that said, if you whitelist IPs or blocks, please submit them to DNSWL.

Thanks for that tip!  I wasn't aware of DNSWL, but will look into it.

Ville