How do I best get SMTP statements logged ?

classic Classic list List threaded Threaded
10 messages Options
K F
Reply | Threaded
Open this post in threaded view
|

How do I best get SMTP statements logged ?

K F

Is it by using debug? How do I set it best to get the smtp statements and their responses?

Reply | Threaded
Open this post in threaded view
|

Re: How do I best get SMTP statements logged ?

Wietse Venema
K F:
> Is it by using debug? How do I set it best to get the smtp statements
> and their responses?

This is not part of routine logging, because it would allow an
adversary to fill your file system with garbage.

Postfix debug logging is for debugging and produces even more output.
That's a way of saying "do not complain about Postfix performance"
if you turn on debug logging.

If you have a business requirement to store SMTP commands routinely,
then I suggest using a network sniffer, or using a reverse proxy
(or load balancer) that does the logging for you. Maybe HaProxy or
Nginx can do that for you.

        Wietse
K F
Reply | Threaded
Open this post in threaded view
|

SV: Re: How do I best get SMTP statements logged ?

K F

Hi Wietse


I will only activate it for our outgoing send array, not for incoming. I know it will take up space, but our customers have expressed some wishes about more knowlegde of the smtp transaction, and apparently can't Seattle for the postfix error messages.

Setting up 10 nginx just for that seem exsessive.


Best regards

Kenneth



---- Wietse Venema skrev ----

K F:

> Is it by using debug? How do I set it best to get the smtp statements

> and their responses?



This is not part of routine logging, because it would allow an

adversary to fill your file system with garbage.



Postfix debug logging is for debugging and produces even more output.

That's a way of saying "do not complain about Postfix performance"

if you turn on debug logging.



If you have a business requirement to store SMTP commands routinely,

then I suggest using a network sniffer, or using a reverse proxy

(or load balancer) that does the logging for you. Maybe HaProxy or

Nginx can do that for you.



Wietse

Reply | Threaded
Open this post in threaded view
|

Re: SV: Re: How do I best get SMTP statements logged ?

Wietse Venema
K F:
> Hi Wietse
>
> I will only activate it for our outgoing send array, not for
> incoming. I know it will take up space, but our customers have
> expressed some wishes about more knowlegde of the smtp transaction,
> and apparently can't Seattle for the postfix error messages.
> Setting up 10 nginx just for that seem exsessive.

Postfix logs the server response. That includes the SMTP status,
the enhanded status code, and the text.  Is that not sufficient?

    status=sent (250 2.0.0 Ok: queued as 8CD1D330BF9)

    status=sent (250 2.0.0 OK 1539347234 g3-v6si942405qvp.214 - gsmtp)

Postfix can report a transcript of the SMTP session, but that
is available only with "sendmail -v":

   -v     Send an email report of the first delivery attempt (Postfix ver-
          sions 2.1 and later). Mail delivery always happens in the  back-
          ground. (only one -v option).

This feature is not available on the SMTP port because it would
need a configuration option like xclient to prevent misuse. If you
really need this for inbound SMTP mail, you can set the DEL_REQ_FLAG_RECORD
flag by changing the source.

            Wietse
Reply | Threaded
Open this post in threaded view
|

Re: SV: Re: How do I best get SMTP statements logged ?

Matus UHLAR - fantomas
In reply to this post by K F
On 12.10.18 17:01, K F wrote:
>I will only activate it for our outgoing send array, not for incoming. I
> know it will take up space, but our customers have expressed some wishes
> about more knowlegde of the smtp transaction, and apparently can't Seattle
> for the postfix error messages.

you can temporarily set debug_peer_list to list of peers you want to debug.

http://www.postfix.org/postconf.5.html#debug_peer_list

would that be enough for you?


>>This is not part of routine logging, because it would allow an
>>adversary to fill your file system with garbage.
>>
>>Postfix debug logging is for debugging and produces even more output.
>>That's a way of saying "do not complain about Postfix performance"
>>if you turn on debug logging.

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
"Where do you want to go to die?" [Microsoft]
K F
Reply | Threaded
Open this post in threaded view
|

Re: SV: Re: How do I best get SMTP statements logged ?

K F
I think that it has to be all receipients.
The problem is that if we get reports about maildelivery it's after the fact, and so far we've not found the problem here, but rather in 'the other end', and thus it makes it more difficult to 'prove' if we don't have smtp actions in the log as well. It's hard to document something that isn't there :-)
Could I add debug for all receipients? Or just turn up the loglevel to verbose so it includes smtp commands?

Den lørdag den 13. oktober 2018 18.40.20 CEST skrev Matus UHLAR - fantomas <[hidden email]>:


On 12.10.18 17:01, K F wrote:
>I will only activate it for our outgoing send array, not for incoming. I
> know it will take up space, but our customers have expressed some wishes
> about more knowlegde of the smtp transaction, and apparently can't Seattle
> for the postfix error messages.

you can temporarily set debug_peer_list to list of peers you want to debug.

http://www.postfix.org/postconf.5.html#debug_peer_list

would that be enough for you?



>>This is not part of routine logging, because it would allow an
>>adversary to fill your file system with garbage.
>>
>>Postfix debug logging is for debugging and produces even more output.
>>That's a way of saying "do not complain about Postfix performance"
>>if you turn on debug logging.


--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
"Where do you want to go to die?" [Microsoft]

Reply | Threaded
Open this post in threaded view
|

Re: SV: Re: How do I best get SMTP statements logged ?

Matus UHLAR - fantomas
On 17.10.18 08:18, K F wrote:
> I think that it has to be all receipients.The problem is that if we get
> reports about maildelivery it's after the fact, and so far we've not found
> the problem here, but rather in 'the other end', and thus it makes it more
> difficult to 'prove' if we don't have smtp actions in the log as well.

simple SMTP log containing somnething like
"status=sent (250 2.0.0 OK 1539769875 s18-v6si13395678wrm.42 - gsmtp)"
"Queued mail for delivery"
"Message 307087686 accepted"

should be enough.


--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
I just got lost in thought. It was unfamiliar territory.
K F
Reply | Threaded
Open this post in threaded view
|

Re: SV: Re: How do I best get SMTP statements logged ?

K F
Hi Matus

I think you are right, that should be enough. What we've seen is some receipients sort of 'goes dark', and they just timeout on the SMTP connection, and the troubleling for us is that it's not 'small companies' that does this.
And getting through to some of the larger mail companies with less than absolute proof is very hard because their first reaction is always 'we don't have a problem', and then our customers think that we have a problem, and so far we've been able to show high likeliness of it not being us, also because there is no change after restart of postfix / server etc. But all of the sudden the problem disappears, sometime after we contact the receipient company, and ask if they have problems.
What should I set to get these extra lines shown?
Den onsdag den 17. oktober 2018 12.08.22 CEST skrev Matus UHLAR - fantomas <[hidden email]>:


On 17.10.18 08:18, K F wrote:
> I think that it has to be all receipients.The problem is that if we get
> reports about maildelivery it's after the fact, and so far we've not found
> the problem here, but rather in 'the other end', and thus it makes it more
> difficult to 'prove' if we don't have smtp actions in the log as well.

simple SMTP log containing somnething like
"status=sent (250 2.0.0 OK 1539769875 s18-v6si13395678wrm.42 - gsmtp)"
"Queued mail for delivery"
"Message 307087686 accepted"

should be enough.



--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.

I just got lost in thought. It was unfamiliar territory.

Reply | Threaded
Open this post in threaded view
|

Re: SV: Re: How do I best get SMTP statements logged ?

Dominic Raferd
On Wed, 17 Oct 2018 at 12:27, K F <[hidden email]> wrote:
... What we've seen is some recipients sort of 'goes dark', and they just timeout on the SMTP connection, and the troubling for us is that it's not 'small companies' that does this.... But all of the sudden the problem disappears, sometime after we contact the recipient company, and ask if they have problems.

Could be their servers are banning your servers for a time e.g. fail2ban? Is it possible some of your users are sending what gets classed as spam, or to non-existent recipients?
K F
Reply | Threaded
Open this post in threaded view
|

Re: SV: Re: How do I best get SMTP statements logged ?

K F
Hi Dominic

Yes, that was my first thought, but they claim our servers wasn't in bad standing in any way, and that it must be on our end. When it gets to be a blame game, logging is gold :-)

Den onsdag den 17. oktober 2018 14.29.06 CEST skrev Dominic Raferd <[hidden email]>:


On Wed, 17 Oct 2018 at 12:27, K F <[hidden email]> wrote:
... What we've seen is some recipients sort of 'goes dark', and they just timeout on the SMTP connection, and the troubling for us is that it's not 'small companies' that does this.... But all of the sudden the problem disappears, sometime after we contact the recipient company, and ask if they have problems.

Could be their servers are banning your servers for a time e.g. fail2ban? Is it possible some of your users are sending what gets classed as spam, or to non-existent recipients?