Despite greylisting result, recipient restrictions still run

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Despite greylisting result, recipient restrictions still run

martin f krafft-2
Hello,

I am doing greylisting in smtpd_client_restrictions and later
a policy server check in smtpd_recipient_restrictions (postconf
included below). smtpd_delay_reject is on (the default).

The weird behaviour I am seeing is that despite a greylisting match
(4xx) in sender restrictions, the recipient restrictions are still
all being evaluated. In the logs, this looks as follows:

  postgrey[27226]: action=greylist, reason=new, client_name=unknown,
    client_address=120.28.68.66, sender=[hidden email],
    recipient=[hidden email]
  postfwd2/policy[1002]: [RULES] rule=3, id=REJECT_HELO_NODNS,
    client=unknown[120.28.68.66], sender=<[hidden email]>,
    recipient=<[hidden email]>, helo=<[120.28.68.66]>,
    proto=ESMTP, state=RCPT, delay=0.00s,
    hits=SET_HELO;SET_NODNS;REJECT_HELO_NODNS, action=REJECT Blocked
    - Suspicious HELO [[120.28.68.66]] and missing reverse DNS
    [120.28.68.66]
  postfix/smtpd[14225]: NOQUEUE: reject: RCPT from
    unknown[120.28.68.66]: 554 5.7.1 <[hidden email]>:
    Recipient address rejected: Blocked - Suspicious HELO
    [[120.28.68.66]] and missing reverse DNS [120.28.68.66];
    from=<[hidden email]> to=<[hidden email]>
    proto=ESMTP helo=<[120.28.68.66]>

This comes as a surprise, especially since the SMTPD_ACCESS_README
says the following about the case when smtpd_delay_reject is on and
all lists are only processed after RCPT TO was sent:

  Restriction lists are still evaluated in the proper order of
  (client, helo, etrn) or (client, helo, sender, relay, recipient,
  data, or end-of-data) restrictions. When a restriction list
  (example: client) evaluates to REJECT or DEFER the restriction
  lists that follow (example: helo, sender, etc.) are skipped.

In my case, the postgrey check in the sender restrictions returns
DEFER, and the README leads me to assume that the recipient list
would be skipped. But it is not.

The end result (5xx) isn't so bad, but I'd still like to understand
what's going on.

Is this behaviour tunable? Or am I doing something wrong?

Thanks!
martin

postconf -n follows:

alias_maps = hash:/etc/aliases
allow_percent_hack = no
append_dot_mydomain = no
biff = no
body_checks = pcre:$config_directory/body_checks.pcre
broken_sasl_auth_clients = no
common_recipient_restrictions = reject_non_fqdn_recipient reject_unknown_recipient_domain reject_unlisted_recipient
common_sender_restrictions = reject_non_fqdn_sender reject_unknown_sender_domain check_sender_mx_access cidr:$config_directory/bogus_mx_addresses.cidr
common_submission_access = permit_mynetworks permit_sasl_authenticated permit_tls_clientcerts
config_directory = /etc/postfix
debug_peer_level = 1
defer_transports = hold
delay_warning_time = 3h
disable_vrfy_command = yes
header_checks = pcre:$config_directory/header_checks.pcre pcre:$config_directory/header_checks_antivirus.pcre
inet_interfaces = all
inet_protocols = all
mailbox_command_maps = hash:$config_directory/mailbox_command_maps.hash
mailbox_transport_maps = hash:$config_directory/mailbox_transport_maps.hash
mailman_destination_recipient_limit = 1
message_size_limit = 26214400
mydomain = madduck.net
mynetworks_style = host
postscreen_access_list = permit_mynetworks
postscreen_blacklist_action = ignore
postscreen_dnsbl_action = ignore
postscreen_greet_action = drop
proxy_read_maps = $canonical_maps $lmtp_generic_maps $local_recipient_maps $mydestination $mynetworks $recipient_bcc_maps $recipient_canonical_maps $relay_domains $relay_recipient_maps $relocated_maps $sender_bcc_maps $sender_canonical_maps $smtp_generic_maps $smtpd_sender_login_maps $transport_maps $virtual_alias_domains $virtual_alias_maps $virtual_mailbox_domains $virtual_mailbox_maps
prxysql = proxy:${sql}
recipient_delimiter = +
relay_clientcerts = hash:$config_directory/relay_ccerts.hash
relay_domains = hash:$config_directory/mailman_lists.hash
relocated_maps = ${prxysql}relocated_maps.pgsql
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtp_tls_fingerprint_digest = sha1
smtp_tls_loglevel = 1
smtp_tls_policy_maps = hash:$config_directory/smtp_tls_policy.hash
smtp_tls_security_level = may
smtpd_client_restrictions = reject_unauth_pipelining check_recipient_access pcre:$config_directory/allow_admin_rcpts.pcre check_client_access pcre:$config_directory/greylist_dialups.pcre
smtpd_data_restrictions =
smtpd_end_of_data_restrictions =
smtpd_etrn_restrictions =
smtpd_helo_required = yes
smtpd_helo_restrictions = reject_invalid_helo_hostname check_recipient_access pcre:$config_directory/allow_admin_rcpts.pcre check_helo_access pcre:$config_directory/reject_helo_myhostname.pcre
smtpd_milters = unix:/clamav/clamav-milter.ctl
smtpd_recipient_restrictions = common_recipient_restrictions check_recipient_access pcre:$config_directory/tarpits.pcre check_recipient_access pcre:$config_directory/allow_admin_rcpts.pcre check_policy_service inet:127.0.0.1:10022 reject_unauth_destination
smtpd_relay_restrictions =
smtpd_restriction_classes = target_greylisting common_submission_access submission_client_restrictions submission_helo_restrictions common_sender_restrictions submission_sender_restrictions common_recipient_restrictions submission_recipient_restrictions
smtpd_sasl_auth_enable = yes
smtpd_sasl_authenticated_header = yes
smtpd_sasl_local_domain = smtprelay.madduck.net
smtpd_sasl_path = private/auth
smtpd_sasl_tls_security_options = noanonymous
smtpd_sasl_type = dovecot
smtpd_sender_restrictions = check_sender_access pcre:$config_directory/blacklisted_senders.pcre check_recipient_access pcre:$config_directory/allow_admin_rcpts.pcre common_sender_restrictions
smtpd_tls_CAfile = /etc/ssl/certs/cacert.org_combined.pem
smtpd_tls_ask_ccert = yes
smtpd_tls_auth_only = yes
smtpd_tls_cert_file = /etc/ssl/certs/$myhostname.pem
smtpd_tls_fingerprint_digest = sha1
smtpd_tls_key_file = /etc/ssl/private/$myhostname.pem
smtpd_tls_received_header = yes
smtpd_tls_security_level = may
soft_bounce = no
sql = pgsql:${config_directory}/
strict_rfc821_envelopes = yes
submission_client_restrictions = common_submission_access
submission_helo_restrictions = common_submission_access
submission_recipient_restrictions = common_recipient_restrictions common_submission_access
submission_sender_restrictions = common_sender_restrictions common_submission_access
target_greylisting = check_policy_service unix:/private/postgrey
transport_maps = ${prxysql}transport_maps.pgsql hash:$config_directory/mailman_lists.hash
virtual_alias_maps = ${prxysql}virtual_alias_maps.pgsql
virtual_gid_maps = ${prxysql}virtual_gid_maps.pgsql
virtual_mailbox_base = /
virtual_mailbox_domains = ${prxysql}virtual_mailbox_domains.pgsql
virtual_mailbox_maps = ${prxysql}virtual_mailbox_maps.pgsql
virtual_minimum_uid = 70000
virtual_uid_maps = ${prxysql}virtual_uid_maps.pgsql

--
@martinkrafft | http://madduck.net/ | http://two.sentenc.es/
 
"for art to exist, for any sort of aesthetic activity or perception to
 exist, a certain physiological precondition is indispensable:
 intoxication."
                                                -- friedrich nietzsche
 
spamtraps: [hidden email]

digital_signature_gpg.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Puting the Postfix's queue into RAM disk

Istvan Prosinger
Hello,

I'll have a project to send 300-400k emails a day from a new IP address
with one server. This can build up a signifficant mail queue on the
server.
We have several similar solutions already working but this time the idea
is to have me do this on a VPS (no SSD drives involved), hmmmm....

What's your verdict about the idea from the subject?

All best,
Istvan

Reply | Threaded
Open this post in threaded view
|

Re: Puting the Postfix's queue into RAM disk

Patrick Ben Koetter-2
* Istvan Prosinger <[hidden email]>:
> Hello,
>
> I'll have a project to send 300-400k emails a day from a new IP
> address with one server. This can build up a signifficant mail queue
> on the server.
> We have several similar solutions already working but this time the
> idea is to have me do this on a VPS (no SSD drives involved),
> hmmmm....

Let's take a look at the numbers and bring them down to something we can
handle in our brains:

400.000,00 day
 16.666,67 hour
    277,78 minute
      4,63 second

Sending 4,63 msg/sec is something regular hardware can do. There's no
technical need to use SSDs or even a tmpfs.

If the machine goes down you risk loosing all messages in delivery. You can
resend them, but then you will have to figure out which ones had already been
delivered and which were still on the machine. I'd wouldn't risk that. It
doesn't pay and you do have the choice to go for something more stable.

p@rick

--
[*] sys4 AG
 
https://sys4.de, +49 (89) 30 90 46 64
Franziskanerstraße 15, 81669 München
 
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
 
Reply | Threaded
Open this post in threaded view
|

Re: Puting the Postfix's queue into RAM disk

Leonardo Rodrigues Magalhães
Em 13/11/15 08:09, Patrick Ben Koetter escreveu:

> * Istvan Prosinger <[hidden email]>:
>> Hello,
>>
>> I'll have a project to send 300-400k emails a day from a new IP
>> address with one server. This can build up a signifficant mail queue
>> on the server.
>> We have several similar solutions already working but this time the
>> idea is to have me do this on a VPS (no SSD drives involved),
>> hmmmm....
> Let's take a look at the numbers and bring them down to something we can
> handle in our brains:
>
> 400.000,00 day
>   16.666,67 hour
>      277,78 minute
>        4,63 second
>
> Sending 4,63 msg/sec is something regular hardware can do. There's no
> technical need to use SSDs or even a tmpfs.
>

     That math fails because usually who send that much emails wants
them to be delivered as fast as possible, so using the whole 24h account
doesnt seem to be the ideal.


--


        Atenciosamente / Sincerily,
        Leonardo Rodrigues
        Solutti Tecnologia
        http://www.solutti.com.br

        Minha armadilha de SPAM, NÃO mandem email
        [hidden email]
        My SPAMTRAP, do not email it



Reply | Threaded
Open this post in threaded view
|

Re: Puting the Postfix's queue into RAM disk

Leonardo Rodrigues Magalhães
In reply to this post by Patrick Ben Koetter-2
Em 13/11/15 08:09, Patrick Ben Koetter escreveu:

> * Istvan Prosinger <[hidden email]>:
>> Hello,
>>
>> I'll have a project to send 300-400k emails a day from a new IP
>> address with one server. This can build up a signifficant mail queue
>> on the server.
>> We have several similar solutions already working but this time the
>> idea is to have me do this on a VPS (no SSD drives involved),
>> hmmmm....
> If the machine goes down you risk loosing all messages in delivery. You can
> resend them, but then you will have to figure out which ones had already been
> delivered and which were still on the machine. I'd wouldn't risk that. It
> doesn't pay and you do have the choice to go for something more stable.
>
>

     I do run a similar system, which deliveres about 150-180k emails
daily, and i do use tmpfs for the postfix spool. To avoid the risk of
losing all data in the case of a system crash, which by the way never
happened on the 3 years the system is running , i use an rsync routine
to sync the tmpfs to the disk. Another script, which runs on the machine
startup, check if the folders are there on the tmpfs and, if they are
not there, creates them and sync from the disk to the tmpfs. The 'sync
from tmpfs to disk' is also configured to run on an eventual system
reboot and will trigger the 'sync from disk to tmpfs' when the system
boots up.



--


        Atenciosamente / Sincerily,
        Leonardo Rodrigues
        Solutti Tecnologia
        http://www.solutti.com.br

        Minha armadilha de SPAM, NÃO mandem email
        [hidden email]
        My SPAMTRAP, do not email it



Reply | Threaded
Open this post in threaded view
|

Re: Puting the Postfix's queue into RAM disk

Istvan Prosinger
In reply to this post by Leonardo Rodrigues Magalhães
On 2015-11-13 12:53, Leonardo Rodrigues wrote:

> Em 13/11/15 08:09, Patrick Ben Koetter escreveu:
>> * Istvan Prosinger <[hidden email]>:
>>> Hello,
>>>
>>> I'll have a project to send 300-400k emails a day from a new IP
>>> address with one server. This can build up a signifficant mail queue
>>> on the server.
>>> We have several similar solutions already working but this time the
>>> idea is to have me do this on a VPS (no SSD drives involved),
>>> hmmmm....
>> Let's take a look at the numbers and bring them down to something we
>> can
>> handle in our brains:
>>
>> 400.000,00 day
>>   16.666,67 hour
>>      277,78 minute
>>        4,63 second
>>
>> Sending 4,63 msg/sec is something regular hardware can do. There's no
>> technical need to use SSDs or even a tmpfs.
>>
>
>     That math fails because usually who send that much emails wants
> them to be delivered as fast as possible, so using the whole 24h
> account doesnt seem to be the ideal.


The point here is that at the start of this, a temporary deferred mail
queue will build up signifficantly pushing most of the load on the file
system, and the idea is to speed up the queue processing to prevent
killing the server (extending the queue size it can process in a time
frame)
My client wouldn't make a problem to deliver the email after a week but
I'm afraid that queue lifetime is out of the question (again, because of
the queue size it would build up)
Reply | Threaded
Open this post in threaded view
|

Re: Puting the Postfix's queue into RAM disk

Harald Koch-2
On 13 November 2015 at 07:51, Istvan Prosinger <[hidden email]> wrote:

The point here is that at the start of this, a temporary deferred mail queue will build up signifficantly pushing most of the load on the file system, and the idea is to speed up the queue processing to prevent killing the server (extending the queue size it can process in a time frame)
My client wouldn't make a problem to deliver the email after a week but I'm afraid that queue lifetime is out of the question (again, because of the queue size it would build up)

IME the only issue with queuing is the fsync() overhead when writing to the queue. For the volumes you're talking about spinning disks might be slow, but SSDs should be just fine. Everything else should already be cached in memory by the operating system - in fact, using a tmpfs "steals" memory that could be used for the OS disk buffer cache.

I think you will buy yourself a lot of (probably unnecessary) pain trying to subvert Postfix's built-in reliability mechanisms.

(you mentioned using a VPS - all the major ones are 100% SSD these days, at least here in North America. My Linode VPS is the fastest machine I have! Also, I would expect modern VPS providers to be running battery-backed caches down at the host layer - which means you'd already be running in what is effectively an all-RAM scenario...)

-- 
Harald
Reply | Threaded
Open this post in threaded view
|

Re: Puting the Postfix's queue into RAM disk

Phil Stracchino
In reply to this post by Istvan Prosinger
On 11/13/15 04:44, Istvan Prosinger wrote:
> Hello,
>
> I'll have a project to send 300-400k emails a day from a new IP address
> with one server. This can build up a signifficant mail queue on the
> server.
> We have several similar solutions already working but this time the idea
> is to have me do this on a VPS (no SSD drives involved), hmmmm....
>
> What's your verdict about the idea from the subject?

Let me pose a single question by way of answer:

"How much do you care about irretrievably losing undelivered mail?"


--
  Phil Stracchino
  Babylon Communications
  [hidden email]
  [hidden email]
  Landline: 603.293.8485
Reply | Threaded
Open this post in threaded view
|

Re: Despite greylisting result, recipient restrictions still run

Viktor Dukhovni
In reply to this post by martin f krafft-2
On Fri, Nov 13, 2015 at 09:12:54PM +1300, martin f krafft wrote:

> I am doing greylisting in smtpd_client_restrictions and later
> a policy server check in smtpd_recipient_restrictions (postconf
> included below). smtpd_delay_reject is on (the default).

Greylisting typically generates a "defer_if_permit" verdict.

    http://www.postfix.org/access.5.html

       DEFER_IF_PERMIT optional text...
              Defer the request if some later restriction would result in a an
              explicit   or    implicit    PERMIT    action.     Reply    with
              "$access_map_defer_code   4.7.1    optional  text..."  when  the
              optional text is specified, otherwise reply with a generic error
              response message.

This is good, because it is silly to defer mail that will ultimately
be rejected.  With "defer_if_permit", processing goes on in the
"hope" that the mail may yet still be rejected by later restrictions.

> The weird behaviour I am seeing is that despite a greylisting match
> (4xx) in sender restrictions, the recipient restrictions are still
> all being evaluated. In the logs, this looks as follows:
>
>   postgrey[27226]: action=greylist, reason=new, client_name=unknown,
>     client_address=120.28.68.66, sender=[hidden email],
>     recipient=[hidden email]

Replies with "defer_if_permit".

>   postfwd2/policy[1002]: [RULES] rule=3, id=REJECT_HELO_NODNS,
>     client=unknown[120.28.68.66], sender=<[hidden email]>,
>     recipient=<[hidden email]>, helo=<[120.28.68.66]>,
>     proto=ESMTP, state=RCPT, delay=0.00s,
>     hits=SET_HELO;SET_NODNS;REJECT_HELO_NODNS, action=REJECT Blocked
>     - Suspicious HELO [[120.28.68.66]] and missing reverse DNS
>     [120.28.68.66]
>   postfix/smtpd[14225]: NOQUEUE: reject: RCPT from
>     unknown[120.28.68.66]: 554 5.7.1 <[hidden email]>:
>     Recipient address rejected: Blocked - Suspicious HELO
>     [[120.28.68.66]] and missing reverse DNS [120.28.68.66];
>     from=<[hidden email]> to=<[hidden email]>
>     proto=ESMTP helo=<[120.28.68.66]>

Excellent, the mail got rejected, rather than deferred.

> In my case, the postgrey check in the sender restrictions returns
> DEFER, and the README leads me to assume that the recipient list
> would be skipped. But it is not.

Does it in fact return "DEFER"?   My money is on "DEFER_IF_PERMIT".

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Despite greylisting result, recipient restrictions still run

martin f krafft-2
also sprach Viktor Dukhovni <[hidden email]> [2015-11-14 05:43 +1300]:
> > I am doing greylisting in smtpd_client_restrictions and later
> > a policy server check in smtpd_recipient_restrictions (postconf
> > included below). smtpd_delay_reject is on (the default).
>
> Greylisting typically generates a "defer_if_permit" verdict.

You are absolutely correct: postgrey returns DEFER_IF_PERMIT by
default. I was looking in the wrong place(s) and didn't consider
that the protocol between postfix and postgrey was more than just
black'n'white.

> >   postgrey[27226]: action=greylist, reason=new, client_name=unknown,
> >     client_address=120.28.68.66, sender=[hidden email],
> >     recipient=[hidden email]
>
> Replies with "defer_if_permit".

Could you discern this from the message or did you just know that
postgrey does this by default and assumed I didn't change that?

--
@martinkrafft | http://madduck.net/ | http://two.sentenc.es/
 
"everyone smiles as you drift past the flower
 that grows so incredibly high."
                                                        -- the beatles
 
spamtraps: [hidden email]

digital_signature_gpg.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Despite greylisting result, recipient restrictions still run

Viktor Dukhovni
On Sat, Nov 14, 2015 at 08:03:41AM +1300, martin f krafft wrote:

> > >   postgrey[27226]: action=greylist, reason=new, client_name=unknown,
> > >     client_address=120.28.68.66, sender=[hidden email],
> > >     recipient=[hidden email]
> >
> > Replies with "defer_if_permit".
>
> Could you discern this from the message or did you just know that
> postgrey does this by default and assumed I didn't change that?

The log message is clearly not explicit about the low-level details
and just says "greylist".

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Puting the Postfix's queue into RAM disk

Istvan Prosinger
In reply to this post by Phil Stracchino
I got two options that I know of. Signifficantly shortening the queue
lifetime, or (not) losing the queue from the RAM disk.
Just trying to measure which is worse (or to hear something new for me)

On 13.11.2015 16:17, Phil Stracchino wrote:

> On 11/13/15 04:44, Istvan Prosinger wrote:
>> Hello,
>>
>> I'll have a project to send 300-400k emails a day from a new IP address
>> with one server. This can build up a signifficant mail queue on the
>> server.
>> We have several similar solutions already working but this time the idea
>> is to have me do this on a VPS (no SSD drives involved), hmmmm....
>>
>> What's your verdict about the idea from the subject?
>
> Let me pose a single question by way of answer:
>
> "How much do you care about irretrievably losing undelivered mail?"
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Puting the Postfix's queue into RAM disk

Ken Simpson
We deliver tens of millions per server per day using SSD spools...
On Fri, Nov 13, 2015 at 11:18 AM Istvan Prosinger <[hidden email]> wrote:
I got two options that I know of. Signifficantly shortening the queue
lifetime, or (not) losing the queue from the RAM disk.
Just trying to measure which is worse (or to hear something new for me)

On 13.11.2015 16:17, Phil Stracchino wrote:
> On 11/13/15 04:44, Istvan Prosinger wrote:
>> Hello,
>>
>> I'll have a project to send 300-400k emails a day from a new IP address
>> with one server. This can build up a signifficant mail queue on the
>> server.
>> We have several similar solutions already working but this time the idea
>> is to have me do this on a VPS (no SSD drives involved), hmmmm....
>>
>> What's your verdict about the idea from the subject?
>
> Let me pose a single question by way of answer:
>
> "How much do you care about irretrievably losing undelivered mail?"
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Puting the Postfix's queue into RAM disk

Istvan Prosinger
Ok. As I mentioned, SSD is not an option on this project. Only RAM or
Raid 10 (shared with other VPSes)

On 13.11.2015 20:56, Ken Simpson wrote:

> We deliver tens of millions per server per day using SSD spools...
> On Fri, Nov 13, 2015 at 11:18 AM Istvan Prosinger <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     I got two options that I know of. Signifficantly shortening the queue
>     lifetime, or (not) losing the queue from the RAM disk.
>     Just trying to measure which is worse (or to hear something new for me)
>
>     On 13.11.2015 16:17, Phil Stracchino wrote:
>      > On 11/13/15 04:44, Istvan Prosinger wrote:
>      >> Hello,
>      >>
>      >> I'll have a project to send 300-400k emails a day from a new IP
>     address
>      >> with one server. This can build up a signifficant mail queue on the
>      >> server.
>      >> We have several similar solutions already working but this time
>     the idea
>      >> is to have me do this on a VPS (no SSD drives involved), hmmmm....
>      >>
>      >> What's your verdict about the idea from the subject?
>      >
>      > Let me pose a single question by way of answer:
>      >
>      > "How much do you care about irretrievably losing undelivered mail?"
>      >
>      >
>
Reply | Threaded
Open this post in threaded view
|

Re: Puting the Postfix's queue into RAM disk

Phil Stracchino
In reply to this post by Istvan Prosinger
On 11/13/15 14:17, Istvan Prosinger wrote:
> I got two options that I know of. Signifficantly shortening the queue
> lifetime, or (not) losing the queue from the RAM disk.
> Just trying to measure which is worse (or to hear something new for me)

If you lack disk space for the queue, the server instance isn't up to
the job in the first place.

If you really think that keeping the queue in RAM is a better option
than on disk, despite that you probably have far more disk than RAM,
consider using a tmpfs instead of a RAMdisk, with the size of the tmpfs
capped at the size RAMdisk you were plannind.  A RAMdisk will use all of
that RAM all of the time, whether needed for queued mail or not.  A
tmpfs will consume only as much RAM at any given time as it needs right
then.



--
  Phil Stracchino
  Babylon Communications
  [hidden email]
  [hidden email]
  Landline: 603.293.8485
Reply | Threaded
Open this post in threaded view
|

Re: Puting the Postfix's queue into RAM disk

Istvan Prosinger


On 13.11.2015 22:53, Phil Stracchino wrote:

> On 11/13/15 14:17, Istvan Prosinger wrote:
>> I got two options that I know of. Signifficantly shortening the queue
>> lifetime, or (not) losing the queue from the RAM disk.
>> Just trying to measure which is worse (or to hear something new for me)
>
> If you lack disk space for the queue, the server instance isn't up to
> the job in the first place.
>
> If you really think that keeping the queue in RAM is a better option
> than on disk, despite that you probably have far more disk than RAM,
> consider using a tmpfs instead of a RAMdisk, with the size of the tmpfs
> capped at the size RAMdisk you were plannind.  A RAMdisk will use all of
> that RAM all of the time, whether needed for queued mail or not.  A
> tmpfs will consume only as much RAM at any given time as it needs right
> then.
>
>
>

Thanks. It's not about a lack of space, puting it that word it's about a
lack of an SSD drive.
Namely, it's about processing speed.
Reply | Threaded
Open this post in threaded view
|

Re: Puting the Postfix's queue into RAM disk

Matthew McGehrin
Is it possible to configure a 2nd VPS instance just for fallback_relay?
That way your primary queue is only for deliveries, and your 2nd
instance can handle the bounces.

I was working for an Online Gaming company and we would deliver 1-2
million messages per day, we had 3 active queues, and 1 delayed queue.

On the active queues we would set the fallback_relay to the delayed
queue, thus ensuring only active mail was being sent first.



Istvan Prosinger wrote:

>
>
> On 13.11.2015 22:53, Phil Stracchino wrote:
>> On 11/13/15 14:17, Istvan Prosinger wrote:
>>> I got two options that I know of. Signifficantly shortening the queue
>>> lifetime, or (not) losing the queue from the RAM disk.
>>> Just trying to measure which is worse (or to hear something new for me)
>>
>> If you lack disk space for the queue, the server instance isn't up to
>> the job in the first place.
>>
>> If you really think that keeping the queue in RAM is a better option
>> than on disk, despite that you probably have far more disk than RAM,
>> consider using a tmpfs instead of a RAMdisk, with the size of the tmpfs
>> capped at the size RAMdisk you were plannind.  A RAMdisk will use all of
>> that RAM all of the time, whether needed for queued mail or not.  A
>> tmpfs will consume only as much RAM at any given time as it needs right
>> then.
>
Reply | Threaded
Open this post in threaded view
|

Re: Puting the Postfix's queue into RAM disk

Istvan Prosinger
The problem is tha there is that one VPS and I wanted 2nd opinions about
my dirty plan.
Thanks

On 2015-11-15 19:03, Matthew McGehrin wrote:

> Is it possible to configure a 2nd VPS instance just for
> fallback_relay? That way your primary queue is only for deliveries,
> and your 2nd instance can handle the bounces.
>
> I was working for an Online Gaming company and we would deliver 1-2
> million messages per day, we had 3 active queues, and 1 delayed queue.
>
> On the active queues we would set the fallback_relay to the delayed
> queue, thus ensuring only active mail was being sent first.
>
>
>
> Istvan Prosinger wrote:
>>
>>
>> On 13.11.2015 22:53, Phil Stracchino wrote:
>>> On 11/13/15 14:17, Istvan Prosinger wrote:
>>>> I got two options that I know of. Signifficantly shortening the
>>>> queue
>>>> lifetime, or (not) losing the queue from the RAM disk.
>>>> Just trying to measure which is worse (or to hear something new for
>>>> me)
>>>
>>> If you lack disk space for the queue, the server instance isn't up to
>>> the job in the first place.
>>>
>>> If you really think that keeping the queue in RAM is a better option
>>> than on disk, despite that you probably have far more disk than RAM,
>>> consider using a tmpfs instead of a RAMdisk, with the size of the
>>> tmpfs
>>> capped at the size RAMdisk you were plannind.  A RAMdisk will use all
>>> of
>>> that RAM all of the time, whether needed for queued mail or not.  A
>>> tmpfs will consume only as much RAM at any given time as it needs
>>> right
>>> then.
>>