advice on securing a transport

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

advice on securing a transport

Eric Abrahamsen
I have a postfix/dovecot installation on the same server as my company's
webapp. This webapp involves a lot of regular data entry, which is a
real pain to do using HTML forms. What I would really like to do is be
able to send structured emails to the server, and have postfix pass them
through a transport to the webapp (a Django site), which would parse the
emails and do CRUD stuff with the database.

I can figure the details out myself, but I'm hoping to get advice on one
particular question: security.

I guess the safest thing would be to require logged-in users: presumably
I could find a way to only accept emails from a local account, but that
would require everyone who had access to this system to have an account
on the server.

The other option would be to maintain a list of authorized email
addresses, and then check incoming messages against this list. This
would be preferable, in that I don't have to bother users to create and
set up (and remember to use) a separate email account. My question is,
is there a truly secure way of only accepting emails from authorized
addresses? Or should I just go with option one and require users to have
accounts?

Any voices of experience/authority very welcome...

Yours,
Eric

Reply | Threaded
Open this post in threaded view
|

Re: advice on securing a transport

Sean Greenslade
On Mon, Sep 05, 2016 at 07:52:02PM +0800, Eric Abrahamsen wrote:

> I have a postfix/dovecot installation on the same server as my company's
> webapp. This webapp involves a lot of regular data entry, which is a
> real pain to do using HTML forms. What I would really like to do is be
> able to send structured emails to the server, and have postfix pass them
> through a transport to the webapp (a Django site), which would parse the
> emails and do CRUD stuff with the database.
>
> I can figure the details out myself, but I'm hoping to get advice on one
> particular question: security.
>
> I guess the safest thing would be to require logged-in users: presumably
> I could find a way to only accept emails from a local account, but that
> would require everyone who had access to this system to have an account
> on the server.
>
> The other option would be to maintain a list of authorized email
> addresses, and then check incoming messages against this list. This
> would be preferable, in that I don't have to bother users to create and
> set up (and remember to use) a separate email account. My question is,
> is there a truly secure way of only accepting emails from authorized
> addresses? Or should I just go with option one and require users to have
> accounts?

Envelope sender / From: field is not to be trusted. Anyone can submit a
message with any envelope sender to an unauthenticated mail server.

I can see two ways of handling this. One is to implement standard
submission port authentication / TLS on this machine, possibly with
virtual users to prevent the need for all users to have local accounts.
The other way is to configure the machine to only accept incoming mail
from your organization's main mail server(s). That way, your regular
mail servers will perform the sender authentication, and then you can
rely on the envelope sender (presuming your main mail servers do not
allow sender spoofing).

--Sean

Reply | Threaded
Open this post in threaded view
|

SV: advice on securing a transport

Sebastian Nielsen
There is possibility to use SPF or DKIM to ensure the sender is not spoofed.
For this particular service, you can run your SPF and/or DKIM validator in
mandatory mode, eg, a missing SPF record will be treated as -all, and a
missing DKIM signature is treated as a invalid one.

Then you can actually use a list of authorized email adresses, even for
third-party operators like GMAIL and such. So if a authorized user, sends a
mail, using a server that is authorized either per that domain's SPF records
or DKIM signature, then the mail will get accepted. Else it will be
rejected.


-----Ursprungligt meddelande-----
Från: [hidden email]
[mailto:[hidden email]] För Sean Greenslade
Skickat: den 5 september 2016 18:36
Till: Eric Abrahamsen <[hidden email]>
Kopia: [hidden email]
Ämne: Re: advice on securing a transport

On Mon, Sep 05, 2016 at 07:52:02PM +0800, Eric Abrahamsen wrote:

> I have a postfix/dovecot installation on the same server as my
> company's webapp. This webapp involves a lot of regular data entry,
> which is a real pain to do using HTML forms. What I would really like
> to do is be able to send structured emails to the server, and have
> postfix pass them through a transport to the webapp (a Django site),
> which would parse the emails and do CRUD stuff with the database.
>
> I can figure the details out myself, but I'm hoping to get advice on
> one particular question: security.
>
> I guess the safest thing would be to require logged-in users:
> presumably I could find a way to only accept emails from a local
> account, but that would require everyone who had access to this system
> to have an account on the server.
>
> The other option would be to maintain a list of authorized email
> addresses, and then check incoming messages against this list. This
> would be preferable, in that I don't have to bother users to create
> and set up (and remember to use) a separate email account. My question
> is, is there a truly secure way of only accepting emails from
> authorized addresses? Or should I just go with option one and require
> users to have accounts?
Envelope sender / From: field is not to be trusted. Anyone can submit a
message with any envelope sender to an unauthenticated mail server.

I can see two ways of handling this. One is to implement standard submission
port authentication / TLS on this machine, possibly with virtual users to
prevent the need for all users to have local accounts.
The other way is to configure the machine to only accept incoming mail from
your organization's main mail server(s). That way, your regular mail servers
will perform the sender authentication, and then you can rely on the
envelope sender (presuming your main mail servers do not allow sender
spoofing).

--Sean



smime.p7s (8K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: advice on securing a transport

lists@lazygranch.com
‎First of all, be wary taking advice from a newbie like me. That said, if you enforce SPF and DKIM in postfix, you will be rejecting a lot of mail. If there is a way to enforce SPF and DKIM on specific senders, that would be another story.

But look at this line from the original message :

"What I would really like to do is be
able to send structured emails to the server, and have postfix pass them
through a transport to the webapp (a Django site), which would parse the
emails and do CRUD stuff with the database.‎"

Normally we read our email from a delivery agent like dovecot, but this mail will, if I understand the objective, with be "machine" read. That step is where you want to enforce SPF and DKIM. 


  Original Message  
From: Sebastian Nielsen
Sent: Monday, September 5, 2016 9:54 AM
To: [hidden email]
Subject: SV: advice on securing a transport

There is possibility to use SPF or DKIM to ensure the sender is not spoofed.
For this particular service, you can run your SPF and/or DKIM validator in
mandatory mode, eg, a missing SPF record will be treated as -all, and a
missing DKIM signature is treated as a invalid one.

Then you can actually use a list of authorized email adresses, even for
third-party operators like GMAIL and such. So if a authorized user, sends a
mail, using a server that is authorized either per that domain's SPF records
or DKIM signature, then the mail will get accepted. Else it will be
rejected.


-----Ursprungligt meddelande-----
Från: [hidden email]
[mailto:[hidden email]] För Sean Greenslade
Skickat: den 5 september 2016 18:36
Till: Eric Abrahamsen <[hidden email]>
Kopia: [hidden email]
Ämne: Re: advice on securing a transport

On Mon, Sep 05, 2016 at 07:52:02PM +0800, Eric Abrahamsen wrote:

> I have a postfix/dovecot installation on the same server as my
> company's webapp. This webapp involves a lot of regular data entry,
> which is a real pain to do using HTML forms. What I would really like
> to do is be able to send structured emails to the server, and have
> postfix pass them through a transport to the webapp (a Django site),
> which would parse the emails and do CRUD stuff with the database.
>
> I can figure the details out myself, but I'm hoping to get advice on
> one particular question: security.
>
> I guess the safest thing would be to require logged-in users:
> presumably I could find a way to only accept emails from a local
> account, but that would require everyone who had access to this system
> to have an account on the server.
>
> The other option would be to maintain a list of authorized email
> addresses, and then check incoming messages against this list. This
> would be preferable, in that I don't have to bother users to create
> and set up (and remember to use) a separate email account. My question
> is, is there a truly secure way of only accepting emails from
> authorized addresses? Or should I just go with option one and require
> users to have accounts?

Envelope sender / From: field is not to be trusted. Anyone can submit a
message with any envelope sender to an unauthenticated mail server.

I can see two ways of handling this. One is to implement standard submission
port authentication / TLS on this machine, possibly with virtual users to
prevent the need for all users to have local accounts.
The other way is to configure the machine to only accept incoming mail from
your organization's main mail server(s). That way, your regular mail servers
will perform the sender authentication, and then you can rely on the
envelope sender (presuming your main mail servers do not allow sender
spoofing).

--Sean


Reply | Threaded
Open this post in threaded view
|

SV: advice on securing a transport

Sebastian Nielsen
No, you're wrong. What the OP should do, is to enforce SPF/DKIM on specific RECEIVERS. For example, enforcing SPF/DKIM on for example [hidden email].

-----Ursprungligt meddelande-----
Från: [hidden email] [mailto:[hidden email]] För [hidden email]
Skickat: den 5 september 2016 19:20
Till: [hidden email]
Ämne: Re: advice on securing a transport

‎First of all, be wary taking advice from a newbie like me. That said, if you enforce SPF and DKIM in postfix, you will be rejecting a lot of mail. If there is a way to enforce SPF and DKIM on specific senders, that would be another story.

But look at this line from the original message :

"What I would really like to do is be
able to send structured emails to the server, and have postfix pass them through a transport to the webapp (a Django site), which would parse the emails and do CRUD stuff with the database.‎"

Normally we read our email from a delivery agent like dovecot, but this mail will, if I understand the objective, with be "machine" read. That step is where you want to enforce SPF and DKIM.


  Original Message
From: Sebastian Nielsen
Sent: Monday, September 5, 2016 9:54 AM
To: [hidden email]
Subject: SV: advice on securing a transport

There is possibility to use SPF or DKIM to ensure the sender is not spoofed.
For this particular service, you can run your SPF and/or DKIM validator in mandatory mode, eg, a missing SPF record will be treated as -all, and a missing DKIM signature is treated as a invalid one.

Then you can actually use a list of authorized email adresses, even for third-party operators like GMAIL and such. So if a authorized user, sends a mail, using a server that is authorized either per that domain's SPF records or DKIM signature, then the mail will get accepted. Else it will be rejected.


-----Ursprungligt meddelande-----
Från: [hidden email]
[mailto:[hidden email]] För Sean Greenslade
Skickat: den 5 september 2016 18:36
Till: Eric Abrahamsen <[hidden email]>
Kopia: [hidden email]
Ämne: Re: advice on securing a transport

On Mon, Sep 05, 2016 at 07:52:02PM +0800, Eric Abrahamsen wrote:

> I have a postfix/dovecot installation on the same server as my
> company's webapp. This webapp involves a lot of regular data entry,
> which is a real pain to do using HTML forms. What I would really like
> to do is be able to send structured emails to the server, and have
> postfix pass them through a transport to the webapp (a Django site),
> which would parse the emails and do CRUD stuff with the database.
>
> I can figure the details out myself, but I'm hoping to get advice on
> one particular question: security.
>
> I guess the safest thing would be to require logged-in users:
> presumably I could find a way to only accept emails from a local
> account, but that would require everyone who had access to this system
> to have an account on the server.
>
> The other option would be to maintain a list of authorized email
> addresses, and then check incoming messages against this list. This
> would be preferable, in that I don't have to bother users to create
> and set up (and remember to use) a separate email account. My question
> is, is there a truly secure way of only accepting emails from
> authorized addresses? Or should I just go with option one and require
> users to have accounts?
Envelope sender / From: field is not to be trusted. Anyone can submit a message with any envelope sender to an unauthenticated mail server.

I can see two ways of handling this. One is to implement standard submission port authentication / TLS on this machine, possibly with virtual users to prevent the need for all users to have local accounts.
The other way is to configure the machine to only accept incoming mail from your organization's main mail server(s). That way, your regular mail servers will perform the sender authentication, and then you can rely on the envelope sender (presuming your main mail servers do not allow sender spoofing).

--Sean




smime.p7s (8K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: SV: advice on securing a transport

Sean Greenslade
On Mon, Sep 05, 2016 at 07:23:10PM +0200, Sebastian Nielsen wrote:
> No, you're wrong. What the OP should do, is to enforce SPF/DKIM on
> specific RECEIVERS. For example, enforcing SPF/DKIM on for example
> [hidden email].

It's important to remember what each step is actually authenticating /
verifying. Both SPF and DKIM verify that a _server_ is authorized to
send mail on behalf of a _domain_. Nothing in either does any sort of
checking / validation of the envelope sender username. Thus, if the
sending mail server allows an authenticated user to send mail as any
envelope sender, that user could send SPF / DKIM valid mail as a sender
they are not authorized to represent.

I have never personally dug into this as it is not an issue for my use
case, but there would need to be some configuration in postfix that
limits which envelope senders are allowed to be used by which user /
vusers in order to ensure full authentication based on envelope sender.

--Sean

Reply | Threaded
Open this post in threaded view
|

Re: advice on securing a transport

lists@lazygranch.com
In reply to this post by Sebastian Nielsen
Seems to me we are in total agreement except for sender versus receiver terminology . That depends on your point of view. But I don't know if you can enforce SPF and DKIM on a domain name basis. If you can't, I assure you much mail will be rejected. Incoming  mail using remailing services will fail SPF.  I'd say I would bounce about 20% of my "desired" email. Probably 75% of the spam. 

I have a 100% failure rate in convincing anyone to fix their SPF and DKIM. Nobody cares because the mail still gets delivered. I couldn't even convince the claws developers that a client which could flag failed DKIM and SPF was worthy. Their solution was look at the header.

  Original Message  
From: Sebastian Nielsen
Sent: Monday, September 5, 2016 10:24 AM
To: [hidden email]
Subject: SV: advice on securing a transport

No, you're wrong. What the OP should do, is to enforce SPF/DKIM on specific RECEIVERS. For example, enforcing SPF/DKIM on for example [hidden email].

-----Ursprungligt meddelande-----
Från: [hidden email] [mailto:[hidden email]] För [hidden email]
Skickat: den 5 september 2016 19:20
Till: [hidden email]
Ämne: Re: advice on securing a transport

‎First of all, be wary taking advice from a newbie like me. That said, if you enforce SPF and DKIM in postfix, you will be rejecting a lot of mail. If there is a way to enforce SPF and DKIM on specific senders, that would be another story.

But look at this line from the original message :

"What I would really like to do is be
able to send structured emails to the server, and have postfix pass them through a transport to the webapp (a Django site), which would parse the emails and do CRUD stuff with the database.‎"

Normally we read our email from a delivery agent like dovecot, but this mail will, if I understand the objective, with be "machine" read. That step is where you want to enforce SPF and DKIM.


Original Message
From: Sebastian Nielsen
Sent: Monday, September 5, 2016 9:54 AM
To: [hidden email]
Subject: SV: advice on securing a transport

There is possibility to use SPF or DKIM to ensure the sender is not spoofed.
For this particular service, you can run your SPF and/or DKIM validator in mandatory mode, eg, a missing SPF record will be treated as -all, and a missing DKIM signature is treated as a invalid one.

Then you can actually use a list of authorized email adresses, even for third-party operators like GMAIL and such. So if a authorized user, sends a mail, using a server that is authorized either per that domain's SPF records or DKIM signature, then the mail will get accepted. Else it will be rejected.


-----Ursprungligt meddelande-----
Från: [hidden email]
[mailto:[hidden email]] För Sean Greenslade
Skickat: den 5 september 2016 18:36
Till: Eric Abrahamsen <[hidden email]>
Kopia: [hidden email]
Ämne: Re: advice on securing a transport

On Mon, Sep 05, 2016 at 07:52:02PM +0800, Eric Abrahamsen wrote:

> I have a postfix/dovecot installation on the same server as my
> company's webapp. This webapp involves a lot of regular data entry,
> which is a real pain to do using HTML forms. What I would really like
> to do is be able to send structured emails to the server, and have
> postfix pass them through a transport to the webapp (a Django site),
> which would parse the emails and do CRUD stuff with the database.
>
> I can figure the details out myself, but I'm hoping to get advice on
> one particular question: security.
>
> I guess the safest thing would be to require logged-in users:
> presumably I could find a way to only accept emails from a local
> account, but that would require everyone who had access to this system
> to have an account on the server.
>
> The other option would be to maintain a list of authorized email
> addresses, and then check incoming messages against this list. This
> would be preferable, in that I don't have to bother users to create
> and set up (and remember to use) a separate email account. My question
> is, is there a truly secure way of only accepting emails from
> authorized addresses? Or should I just go with option one and require
> users to have accounts?

Envelope sender / From: field is not to be trusted. Anyone can submit a message with any envelope sender to an unauthenticated mail server.

I can see two ways of handling this. One is to implement standard submission port authentication / TLS on this machine, possibly with virtual users to prevent the need for all users to have local accounts.
The other way is to configure the machine to only accept incoming mail from your organization's main mail server(s). That way, your regular mail servers will perform the sender authentication, and then you can rely on the envelope sender (presuming your main mail servers do not allow sender spoofing).

--Sean



Reply | Threaded
Open this post in threaded view
|

SV: SV: advice on securing a transport

Sebastian Nielsen
In reply to this post by Sean Greenslade
LazyGranch:
I look it at the point of view of the server who are receiving the mail.
So basically, the OP has some email adress like "[hidden email]"
that receives mail and processes this automatically into a database.

Only authorized users are allowed to send to this specifically crafted email
adress.

Thus, the receiving postfix server, could be configured to add a pass/fail
header of SPF and DKIM authentication.
Then the program acting on transport (eg, the actual /usr/bin program that
is configured as transport destination for [hidden email]) just
checks this header. If not at least one of them is PASS and the Return-Path:
header matches whats on a authorized list, the program could be configured
to just ignore the received mail in question.

Care needs to be taken so not anyone can fool the validation by inserting a
fraudulent SPF or DKIM header, which would result in a duplicate, one
genuine and one fake header.
This can be accomplished by either checking for duplicate headers and
failing authentication if there is duplicate SPF or DKIM header. (note:
DKIM-header = The header with the validation result, inserted by the local
validator, NOT the actual signature).
Or you can configure the validation process to always purge out any existing
validation headers before inserting its own.

Thus, actually, the postfix server does not need to reject any mail, this
could be coded into the transport program which also does all the
modification to the django app database, to dump all unauthenticated (eg, no
valid SPF or DKIM) and unauthorized (not on authorized list) into /dev/null.


Sean Greenslade:
Thats the responsibility of the server who is authorized to act on behalf of
that domain.



smime.p7s (8K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: advice on securing a transport

lists@lazygranch.com
‎"Thus, the receiving postfix server, could be configured to add a pass/fail
header of SPF and DKIM authentication."

This came up a few months ago on the list, with the idea of doing a rewrite on the subject line. For example, SpamAssassin writes "spam". The new rewrite would indicate SPF and DKIM failures. Nobody came up with a turnkey solution to this, but I for one would like to have this, since I don't have a client that does this automatically. 

Supposedly there is a plugin for Thunderbird email that reads the header and does such notification, but I would trust a postfix implementation more.

  Original Message  
From: Sebastian Nielsen
Sent: Monday, September 5, 2016 11:18 AM
To: [hidden email]
Subject: SV: SV: advice on securing a transport

LazyGranch:
I look it at the point of view of the server who are receiving the mail.
So basically, the OP has some email adress like "[hidden email]"
that receives mail and processes this automatically into a database.

Only authorized users are allowed to send to this specifically crafted email
adress.

Thus, the receiving postfix server, could be configured to add a pass/fail
header of SPF and DKIM authentication.
Then the program acting on transport (eg, the actual /usr/bin program that
is configured as transport destination for [hidden email]) just
checks this header. If not at least one of them is PASS and the Return-Path:
header matches whats on a authorized list, the program could be configured
to just ignore the received mail in question.

Care needs to be taken so not anyone can fool the validation by inserting a
fraudulent SPF or DKIM header, which would result in a duplicate, one
genuine and one fake header.
This can be accomplished by either checking for duplicate headers and
failing authentication if there is duplicate SPF or DKIM header. (note:
DKIM-header = The header with the validation result, inserted by the local
validator, NOT the actual signature).
Or you can configure the validation process to always purge out any existing
validation headers before inserting its own.

Thus, actually, the postfix server does not need to reject any mail, this
could be coded into the transport program which also does all the
modification to the django app database, to dump all unauthenticated (eg, no
valid SPF or DKIM) and unauthorized (not on authorized list) into /dev/null.


Sean Greenslade:
Thats the responsibility of the server who is authorized to act on behalf of
that domain.


Reply | Threaded
Open this post in threaded view
|

Re: advice on securing a transport

Sean Greenslade
In reply to this post by Sebastian Nielsen
On Mon, Sep 05, 2016 at 08:17:40PM +0200, Sebastian Nielsen wrote:
> Sean Greenslade:
> Thats the responsibility of the server who is authorized to act on behalf of
> that domain.

Yes, however I am trying to make this discussion relevant to the OP's
question. Authenticating based solely on originating server puts the
authentication job onto that originating server. If that server allows
users to send mail as other users on the same domain, that is a
potential security hole, since the email parsing server does not know
what user auth'd to the mail server, only that _someone_ did.

It may be that user spoofing isn't an issue. If it's not, then this
doesn't matter. But if the OP wants to, say, only allow certain users to
send messages to this parser, they must verify that the mail server
restricts envelope sender based on authenticated user.

--Sean

Reply | Threaded
Open this post in threaded view
|

Re: advice on securing a transport

Eric Abrahamsen
Sean Greenslade <[hidden email]> writes:

> On Mon, Sep 05, 2016 at 08:17:40PM +0200, Sebastian Nielsen wrote:
>> Sean Greenslade:
>> Thats the responsibility of the server who is authorized to act on behalf of
>> that domain.
>
> Yes, however I am trying to make this discussion relevant to the OP's
> question. Authenticating based solely on originating server puts the
> authentication job onto that originating server. If that server allows
> users to send mail as other users on the same domain, that is a
> potential security hole, since the email parsing server does not know
> what user auth'd to the mail server, only that _someone_ did.
>
> It may be that user spoofing isn't an issue. If it's not, then this
> doesn't matter. But if the OP wants to, say, only allow certain users to
> send messages to this parser, they must verify that the mail server
> restricts envelope sender based on authenticated user.

Thanks to all of you for your responses! Looks like this is more
complicated than I thought. A couple of things:

1. I should have said that server-side user accounts are all virtual --
   postfix authenticates against the dovecot user database. Adding new
   user accounts isn't terribly onerous, the main thing would be getting
   my users to use them. No one wants to set up a new email account just
   to operate this thing.
2. Their usual email address are all hotmail, gmail, sina.cn, etc. Major
   freemail providers. Simply validating SPF against the domain won't
   really do that much.

So maybe I should just give up and make them create local accounts. This
system should be able to delete data from the database, so... I'd like
to be a little bit paranoid.

I also realized that I have a mailman installation on this same server,
running as a transport on the same postfix installation, and I'd like to
see how it does it. My guess is it fluffs on security, but let's see...

Thanks again,
Eric