PostFix with Postgrey (large mail queue)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

PostFix with Postgrey (large mail queue)

Amaru Netapshaak

Greetings Postfix-Users!!

I recently setup Postfix with a slew of other apps to fight spam, and I am using postgrey for greylisting.

I've noticed that after a few days, I have 4000+ email messages in the queue (as shown with
"mailq" command).  And the queue is growing.

I presume this must be something to do with greylisting.  I am receiving email as normal, and everything
seems functional.  I just want to be sure that having this many messages in a queue is normal and wont
cause problems down the road. All messages in the queue are addressed to foreign hosts and most
have "Operation Timed Out"

Should I be removing these?  My instinct tells me yes! 

For example:

F0F8AB18271  4871 Sun Aug 10 03:13:11  MAILER-DAEMON
       (connect to <host removed>[<ip removed>]:25: Operation timed out)
                         [hidden email]

-- 29608 Kbytes in 4185 Requests.

Thanks for any and all advice!

++ Amaru

Reply | Threaded
Open this post in threaded view
|

Re: PostFix with Postgrey (large mail queue)

Noel Jones-2
Amaru Netapshaak wrote:

>
> Greetings Postfix-Users!!
>
> I recently setup Postfix with a slew of other apps to fight spam, and I
> am using postgrey for greylisting.
>
> I've noticed that after a few days, I have 4000+ email messages in the
> queue (as shown with
> "mailq" command).  And the queue is growing.
>
> I presume this must be something to do with greylisting.  I am receiving
> email as normal, and everything
> seems functional.  I just want to be sure that having this many messages
> in a queue is normal and wont
> cause problems down the road. All messages in the queue are addressed to
> foreign hosts and most
> have "Operation Timed Out"
>
> Should I be removing these?  My instinct tells me yes!
>
> For example:
>
> F0F8AB18271  4871 Sun Aug 10 03:13:11  MAILER-DAEMON
>        (connect to <host removed>[<ip removed>]:25: Operation timed out)
>                          [hidden email]
>
> -- 29608 Kbytes in 4185 Requests.
>
> Thanks for any and all advice!
>
> ++ Amaru
>

Yes, it's usually very bad to have a queue full of
MAILER-DAEMON messages.

What are these, NDR's to undeliverable/unknown/invalid recipients?
(use "postcat -q F0F8AB18271" to view the message)
If so, don't accept mail to undeliverable recipients.

Accepting and then bouncing mail to unknown recipients will
greatly increase load on your server, and will get you
blacklisted.  Instead, configure your system to reject unknown
recipients during SMTP.

Here's some places to start:
http://www.postfix.org/ADDRESS_CLASS_README.html
http://www.postfix.org/LOCAL_RECIPIENT_README.html

If these are not NDR's, please give us more information
http://www.postfix.org/DEBUG_README.html#mail

--
Noel Jones
Reply | Threaded
Open this post in threaded view
|

Re: PostFix with Postgrey (large mail queue)

Amaru Netapshaak
In reply to this post by Amaru Netapshaak
>Yes, it's usually very bad to have a queue full of

>MAILER-DAEMON messages.
>
>What are these, NDR's to undeliverable/unknown/invalid recipients?
>(use "postcat -q F0F8AB18271" to view the message)
>If so, don't accept mail to undeliverable recipients.
>
>Accepting and then bouncing mail to unknown recipients will
>greatly increase load on your server, and will get you
>blacklisted.  Instead, configure your system to reject unknown
>recipients during SMTP.
>
>Here's some places to start:
>http://www.postfix.org/ADDRESS_CLASS_README.html
>http://www.postfix.org/LOCAL_RECIPIENT_README.html
>
>If these are not NDR's, please give us more information
>http://www.postfix.org/DEBUG_README.html#mail.




Noel,

Thanks for your response.  The emails in the queue are bounces
from my email server.  This box is just an email gateway which relays
to an internal email server.

I believe NDR means "Non Deliverable Return" -- im basing my decisions
on that assumption. :)   So yes, these are NDRs. 

All the MAILER-DAEMON messages are for recipients on my internal
email server, but those recipients no longer exist.  There are thousands
of accounts on the internal email server, so specifying them in a
recipient list seems excessive.  Perhaps im not reading those documents
carefully enough.  I am already specifying $relay_domains.  But since
I am relaying, using "local_recipient_maps" wont help, and a "relay_
recipient_map" would be a major hassle.  Any other tips?

It appears that my email gateway is attempting to bounce a message
back to the original sender that the recipient no longer exists. I can
attest that all recipients of the bounce email are not valid servers either
and are spam.  I cannot telnet to these systems on 25 or 110.. so thats
why they are timing out on my gateway.

Any additional information is greatly appreciated!

Thank you!

++ Amaru



Reply | Threaded
Open this post in threaded view
|

Re: PostFix with Postgrey (large mail queue)

Noel Jones-2
Amaru Netapshaak wrote:

> Thanks for your response.  The emails in the queue are bounces
> from my email server.  This box is just an email gateway which relays
> to an internal email server.
>
> I believe NDR means "Non Deliverable Return" -- im basing my decisions
> on that assumption. :)   So yes, these are NDRs.
>
> All the MAILER-DAEMON messages are for recipients on my internal
> email server, but those recipients no longer exist.  There are thousands
> of accounts on the internal email server, so specifying them in a
> recipient list seems excessive.  Perhaps im not reading those documents
> carefully enough.  I am already specifying $relay_domains.  But since
> I am relaying, using "local_recipient_maps" wont help, and a "relay_
> recipient_map" would be a major hassle.  Any other tips?

You need to validate recipients.  The best way to do that is
to create a relay_recipient_maps table.  Automate the
procedure so you don't have to mess with updates.

The trouble of implementing proper recipient validation will
pay off handsomely when compared with dealing with an
overloaded server.  And you will be blacklisted as a
backscatter source, which will be a major headache.

If it's simply not possible to get a user list from the
internal server, let postfix build one automatically by using
recipient verification.  This only works if the internal
server rejects unknown recipients during SMTP, and doesn't
itself accept and bounce.
http://www.postfix.org/ADDRESS_VERIFICATION_README.html#recipient

--
Noel Jones
Reply | Threaded
Open this post in threaded view
|

Re: PostFix with Postgrey (large mail queue)

Benny Pedersen
In reply to this post by Amaru Netapshaak

On Tue, August 12, 2008 03:11, Amaru Netapshaak wrote:
> Any additional information is greatly appreciated!

postconf -n is needed anyway

--
Benny Pedersen
Need more webspace ? http://www.servage.net/?coupon=cust37098

Reply | Threaded
Open this post in threaded view
|

Re: PostFix with Postgrey (large mail queue)

Amaru Netapshaak
In reply to this post by Noel Jones-2
Hello,

I have implemented the "recipient_relay_map" in main.cf, and have my
email server & gateway automatically building a fresh recipient hash
every hour..  yet, it doesnt work.   If I create an account on my internal
email server, and dont put that address in my hash, the account can still
receive email.

Here is my postconf -n  info:

biff = no
command_directory = /usr/sbin
config_directory = /etc/postfix
daemon_directory = /usr/libexec/postfix
data_directory = /var/lib/postfix
default_destination_concurrency_limit = 20
default_privs = nobody
fast_flush_domains = $relay_domains
html_directory = no
in_flow_delay = 1s
inet_interfaces = all
local_destination_concurrency_limit = 5
local_recipient_maps =
local_transport = local
mail_owner = postfix
mailq_path = /usr/bin/mailq
manpage_directory = /usr/local/man
mydestination = $myhostname, XXX.com, localhost, localhost.$mydomain
mydomain = XXX.com
myhostname = XXX.com
mynetworks = XXX.XXX.XXX.XXX/24, 127.0.0.0/8
myorigin = $myhostname
newaliases_path = /usr/bin/newaliases
notify_classes = protocol, resource, software
queue_directory = /var/spool/postfix
readme_directory = no
relay_domains = hash:/etc/postfix/relay_domains
relay_recipient_maps = hash:/etc/postfix/relay_recipients
sample_directory = /etc/postfix
sendmail_path = /usr/sbin/sendmail
setgid_group = postdrop
smtpd_banner = $myhostname ESMTP
smtpd_recipient_restrictions = permit_mynetworks, reject_unauth_destination, reject_non_fqdn_recipient, check_policy_service inet:127.0.0.1:10023
transport_maps = hash:/etc/postfix/transport
unknown_local_recipient_reject_code = 550

I have obfuscated my hostnames and IP addresses for security.

My instinct tells me this has something to do with the fact that im specifying
"relay_domains" AND "recipient_relay_maps", but maybe im missing
somethign in the documentation?

Any assistance is greatly appreciated.

Thank you!

++ Amaru


--- On Tue, 8/12/08, Noel Jones <[hidden email]> wrote:
From: Noel Jones <[hidden email]>
Subject: Re: PostFix with Postgrey (large mail queue)
To: "Amaru Netapshaak" <[hidden email]>
Cc: [hidden email]
Date: Tuesday, August 12, 2008, 2:57 AM

Amaru Netapshaak wrote:
> Thanks for your response. The emails in the queue are bounces
> from my email server. This box is just an email gateway which relays
> to an internal email server.
>
> I believe NDR means "Non Deliverable Return" -- im basing my
decisions
> on that assumption. :) So yes, these are NDRs.
>
> All the MAILER-DAEMON messages are for recipients on my internal
> email server, but those recipients no longer exist. There are thousands
> of accounts on the internal email server, so specifying them in a
> recipient list seems excessive. Perhaps im not reading those documents
> carefully enough. I am already specifying $relay_domains. But since
> I am relaying, using "local_recipient_maps" wont help, and a
"relay_
> recipient_map" would be a major hassle. Any other tips?

You need to validate recipients. The best way to do that is
to create a relay_recipient_maps table. Automate the
procedure so you don't have to mess with updates.

The trouble of implementing proper recipient validation will
pay off handsomely when compared with dealing with an
overloaded server. And you will be blacklisted as a
backscatter source, which will be a major headache.

If it's simply not possible to get a user list from the
internal server, let postfix build one automatically by using
recipient verification. This only works if the internal
server rejects unknown recipients during SMTP, and doesn't
itself accept and bounce.
http://www.postfix.org/ADDRESS_VERIFICATION_README.html#recipient

--
Noel Jones

Reply | Threaded
Open this post in threaded view
|

Re: PostFix with Postgrey (large mail queue)

Noel Jones-2
Amaru Netapshaak wrote:

> Hello,
>
> I have implemented the "recipient_relay_map" in main.cf, and have my
> email server & gateway automatically building a fresh recipient hash
> every hour..  yet, it doesnt work.   If I create an account on my internal
> email server, and dont put that address in my hash, the account can still
> receive email.
>
> Here is my postconf -n  info:
>
> biff = no
> command_directory = /usr/sbin
> config_directory = /etc/postfix
> daemon_directory = /usr/libexec/postfix
> data_directory = /var/lib/postfix
> default_destination_concurrency_limit = 20
> default_privs = nobody
> fast_flush_domains = $relay_domains
> html_directory = no
> in_flow_delay = 1s
> inet_interfaces = all
> local_destination_concurrency_limit = 5
> local_recipient_maps =
> local_transport = local
> mail_owner = postfix
> mailq_path = /usr/bin/mailq
> manpage_directory = /usr/local/man
> mydestination = $myhostname, XXX.com, localhost, localhost.$mydomain
> mydomain = XXX.com
> myhostname = XXX.com
> mynetworks = XXX.XXX.XXX.XXX/24, 127.0.0.0/8
> myorigin = $myhostname
> newaliases_path = /usr/bin/newaliases
> notify_classes = protocol, resource, software
> queue_directory = /var/spool/postfix
> readme_directory = no
> relay_domains = hash:/etc/postfix/relay_domains
> relay_recipient_maps = hash:/etc/postfix/relay_recipients
> sample_directory = /etc/postfix
> sendmail_path = /usr/sbin/sendmail
> setgid_group = postdrop
> smtpd_banner = $myhostname ESMTP
> smtpd_recipient_restrictions = permit_mynetworks,
> reject_unauth_destination, reject_non_fqdn_recipient,
> check_policy_service inet:127.0.0.1:10023
> transport_maps = hash:/etc/postfix/transport
> unknown_local_recipient_reject_code = 550
>
> I have obfuscated my hostnames and IP addresses for security.
>
> My instinct tells me this has something to do with the fact that im
> specifying
> "relay_domains" AND "recipient_relay_maps", but maybe im missing
> somethign in the documentation?

[please don't top post]

No, you must specify both relay_domains - to tell postfix
which domains to relay for - and relay_recipient_maps - to
tell postfix which users are valid in those domains.

Recipient validation can be broken by wildcard entries in
sender_canonical_maps, recipient_canonical_maps,
canonical_maps or virtual_alias_maps, or by a domain wildcard
in relay_recipient_maps.

It doesn't appear you are using *canonical_maps or
virtual_alias_maps, so I assume you have a wildcard entry in
your relay_recipient_maps.  Don't do that.
http://www.postfix.org/postconf.5.html#relay_recipient_maps

If you need more help, show unmodified logs.
http://www.postfix.org/DEBUG_README.html#mail

--
Noel Jones
Reply | Threaded
Open this post in threaded view
|

Re: PostFix with Postgrey (large mail queue)

Victor Duchovni
On Fri, Sep 05, 2008 at 12:00:26PM -0500, Noel Jones wrote:

> >mydestination = $myhostname, XXX.com, localhost, localhost.$mydomain
> >relay_domains = hash:/etc/postfix/relay_domains
> >relay_recipient_maps = hash:/etc/postfix/relay_recipients
> >
> >My instinct tells me this has something to do with the fact that im
> >specifying
> >"relay_domains" AND "recipient_relay_maps", but maybe im missing
> >somethign in the documentation?
>
> [please don't top post]
>
> No, you must specify both relay_domains - to tell postfix
> which domains to relay for - and relay_recipient_maps - to
> tell postfix which users are valid in those domains.
>
> Recipient validation can be broken by wildcard entries in
> sender_canonical_maps, recipient_canonical_maps,
> canonical_maps or virtual_alias_maps, or by a domain wildcard
> in relay_recipient_maps.
>
> It doesn't appear you are using *canonical_maps or
> virtual_alias_maps, so I assume you have a wildcard entry in
> your relay_recipient_maps.  Don't do that.
> http://www.postfix.org/postconf.5.html#relay_recipient_maps

And of course recipient validation for domains listed in $mydestination
is done via local_recipient_maps.

--
        Viktor.

Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.

To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:[hidden email]?body=unsubscribe%20postfix-users>

If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.