Are there plans for a build-in support of REDIS-tables?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Are there plans for a build-in support of REDIS-tables?

kris_h
This post was updated on .
Hello Wietse,
hello all,

yes, there is a related msg -
http://postfix.1071664.n5.nabble.com/Redis-Plugin-td87004.html - but dated
2016 and it kept unanswered...

I'm looking for something similar like:
http://www.postfix.org/memcache_table.5.html

I would prefer redis over memcache, since it supports
replication/syncronisation over multiple servers.

Thank a lot in advance

Kris



--
Sent from: http://postfix.1071664.n5.nabble.com/Postfix-Users-f2.html
Reply | Threaded
Open this post in threaded view
|

Re: Are there plans for a buld-in support of REDIS-tables?

Ansgar Wiechers
On 2020-01-09 kris_h wrote:
> yes, there is a related msg -
> http://postfix.1071664.n5.nabble.com/Redis-Plugin-td87004.html - but dated
> 2016 and it kept unanswered...
>
> I'm looking for something similar like:
> http://www.postfix.org/memcache_table.5.html
>
> I would prefer redis over memcache, since it supports
> replication/syncronisation over multiple servers.

I would recommend using a configuration management system like Puppet,
Ansible, Chef, ... for deploying tables across multiple servers instead
of replicating the information with something like Redis.

Regards
Ansgar Wiechers
--
"Abstractions save us time working, but they don't save us time learning."
--Joel Spolsky
Reply | Threaded
Open this post in threaded view
|

Re: Are there plans for a buld-in support of REDIS-tables?

kris_h

Hey Ansgar,

thank you for your quick reply.

For sure the quasi-static tables should be managed via deploymant-systems or
simplier rsync and that alike.

We distribute the more dynamic tables - e.g. cidr-tables with self-harvested
current spammer's IPs - actually by simply distributing those files with
rsync.

I hesitate to use distributed filesystems, because the multiple r/w-access
results in big overhead/load.
Also the 'permanent' inbound MTA-avaibility goes miles before applying
rules/filtering.

I searching for pros/cons for:
1. distributing new entries to all servers with special program/interfaces
to named pipes or memcache
2. DRBD or GlusterFS
3. REDIS - which allows triggers

Maybe Wietse has a redis-table-module in his drawer? ;-)

Kris



--
Sent from: http://postfix.1071664.n5.nabble.com/Postfix-Users-f2.html
Reply | Threaded
Open this post in threaded view
|

Re: Are there plans for a buld-in support of REDIS-tables?

Wietse Venema
kris_h:

>
> Hey Ansgar,
>
> thank you for your quick reply.
>
> For sure the quasi-static tables should be managed via deploymant-systems or
> simplier rsync and that alike.
>
> We distribute the more dynamic tables - e.g. cidr-tables with self-harvested
> current spammer's IPs - actually by simply distributing those files with
> rsync.
>
> I hesitate to use distributed filesystems, because the multiple r/w-access
> results in big overhead/load.
> Also the 'permanent' inbound MTA-avaibility goes miles before applying
> rules/filtering.
>
> I searching for pros/cons for:
> 1. distributing new entries to all servers with special program/interfaces
> to named pipes or memcache
> 2. DRBD or GlusterFS
> 3. REDIS - which allows triggers
>
> Maybe Wietse has a redis-table-module in his drawer? ;-)

Sorry, I don't write code and then sadistically keep it for myself.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Are there plans for a buld-in support of REDIS-tables?

A. Schulze
In reply to this post by kris_h


Am 09.01.20 um 17:12 schrieb kris_h:
> We distribute the more dynamic tables - e.g. cidr-tables with self-harvested
> current spammer's IPs - actually by simply distributing those files with
> rsync.

we use an rbldnsd to build and serve an internal zone with similar data.
Usual DNS lookups are done by postfix (reject_rbl_client and reject_rhsbl_client)
it's fast enough (for our use-cases)

Andreas
Reply | Threaded
Open this post in threaded view
|

Re: Are there plans for a buld-in support of REDIS-tables?

Fred Morris
In reply to this post by kris_h
1) Somebody suggests using e.g. Ansible to update static tables; but that
requires restarting or triggering a reload in some fashion, does it not?

2) Plus there is the matter of querying your inventory to see what the
configurations are on all nodes (what if they're intentionally not all the
same?).

Key value stores (Redis, etcd, I've used DNS, there are others) are an
option which address both issues, if network connected. (If a file can be
updated over the network, is the filesystem really "not connected to the
network"?)

I would still like to understand what the guiding philosophy for network
connected maps is other than "bad because security", and what the real
attack surface/security concerns are. For this application and particular
implementation, not the planet... maybe for particular choices of
implementation tools, for instance I know that Postfix relies on some
"primitives" to perform "risky" operations and that's easily cast as
defensive coding based on defensive design (against the implementation
language).

I may choose to disable certain security checks and recompile, but I'd
like to know what application security assumptions I'm violating with more
specificity than "trying to connect to localhost": what attack from the
application's vantage accrues to localhost which does not accrue to a unix
domain socket for example? I'm willing to get theoretical enough to ask:
are all of these assumptions still valid in an immutable and containerized
world (what role does partitioning of roles and privileges play)?

I'm waiting patiently for that discussion.

(If there's another way, even hackerish, to call a networked service to
validate aliases at SMTP handshake which doesn't require the equally
hackerish measure of disabling security checks and recompiling then I'd
love to hear about it.)

--

Fred Morris

Reply | Threaded
Open this post in threaded view
|

Re: Are there plans for a buld-in support of REDIS-tables?

Phillip Schichtel

My suggestion: Bridge tcp or socketmap requests to what ever
database/remote-service you like. I'm using that approach to bridge lookups
to a rest web service for a while now using a small application [1] (built
by me, documentation is light, but not a lot of code). I have that tool
bound to 127.0.0.1 and outgoing requests are using TLS. I hope that helps.

~ Phillip Schichtel

[1]: https://github.com/pschichtel/postfix-rest-connector

Am 9. Januar 2020 18:33:11 schrieb Fred Morris <[hidden email]>:

> 1) Somebody suggests using e.g. Ansible to update static tables; but that
> requires restarting or triggering a reload in some fashion, does it not?
>
> 2) Plus there is the matter of querying your inventory to see what the
> configurations are on all nodes (what if they're intentionally not all the
> same?).
>
> Key value stores (Redis, etcd, I've used DNS, there are others) are an
> option which address both issues, if network connected. (If a file can be
> updated over the network, is the filesystem really "not connected to the
> network"?)
>
> I would still like to understand what the guiding philosophy for network
> connected maps is other than "bad because security", and what the real
> attack surface/security concerns are. For this application and particular
> implementation, not the planet... maybe for particular choices of
> implementation tools, for instance I know that Postfix relies on some
> "primitives" to perform "risky" operations and that's easily cast as
> defensive coding based on defensive design (against the implementation
> language).
>
> I may choose to disable certain security checks and recompile, but I'd
> like to know what application security assumptions I'm violating with more
> specificity than "trying to connect to localhost": what attack from the
> application's vantage accrues to localhost which does not accrue to a unix
> domain socket for example? I'm willing to get theoretical enough to ask:
> are all of these assumptions still valid in an immutable and containerized
> world (what role does partitioning of roles and privileges play)?
>
> I'm waiting patiently for that discussion.
>
> (If there's another way, even hackerish, to call a networked service to
> validate aliases at SMTP handshake which doesn't require the equally
> hackerish measure of disabling security checks and recompiling then I'd
> love to hear about it.)
>
> --
>
> Fred Morris



Reply | Threaded
Open this post in threaded view
|

Re: Are there plans for a buld-in support of REDIS-tables?

Wietse Venema
In reply to this post by Fred Morris
Fred Morris:
> 1) Somebody suggests using e.g. Ansible to update static tables; but that
> requires restarting or triggering a reload in some fashion, does it not?

Nope. Only really incompetent systems would copy a new file OVER
an existing file. Competent systems use 'rename'. Postfix programs
will close the old table and restart at their convenience.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: Are there plans for a buld-in support of REDIS-tables?

Michael Ströder
In reply to this post by kris_h
On 1/9/20 5:12 PM, kris_h wrote:
> We distribute the more dynamic tables - e.g. cidr-tables with self-harvested
> current spammer's IPs - actually by simply distributing those files with
> rsync.
> [..]
> I searching for pros/cons for:

postfix supports LDAP lookups out-of-the-box.

Is using LDAP an option for you?

If yes, setting up a couple of OpenLDAP replicas is not that hard.

Ciao, Michael.