any api to read logs ?

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

any api to read logs ?

Илья Шипицин
Hello,


we use automation to send messages.
from the automation point of view it is nice to know what happened to message

I think about the following

1) automation send email via smtp --> id of message
2) automation ask postfix via (rest) api "hey, tell me history of message id ..."

any suggestion ?

Ilya Shipitsin
Reply | Threaded
Open this post in threaded view
|

Re: any api to read logs ?

Wietse Venema
???? ???????:

> Hello,
>
> we use automation to send messages.
> from the automation point of view it is nice to know what happened to
> message
>
> I think about the following
>
> 1) automation send email via smtp --> id of message
> 2) automation ask postfix via (rest) api "hey, tell me history of message
> id ..."
>
> any suggestion ?

Open a web search engine, ask for for 'logfile analysis tools'.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: any api to read logs ?

Илья Шипицин
logfile analys is good for human, it is not rest api.
I did search already

any rest api ?

сб, 29 сент. 2018 г. в 16:21, Wietse Venema <[hidden email]>:
???? ???????:
> Hello,
>
> we use automation to send messages.
> from the automation point of view it is nice to know what happened to
> message
>
> I think about the following
>
> 1) automation send email via smtp --> id of message
> 2) automation ask postfix via (rest) api "hey, tell me history of message
> id ..."
>
> any suggestion ?

Open a web search engine, ask for for 'logfile analysis tools'.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: any api to read logs ?

Matus UHLAR - fantomas
On 29.09.18 16:42, Илья Шипицин wrote:
>logfile analys is good for human, it is not rest api.

postfix produces logs, does not read nor parse them.

>I did search already
>
>any rest api ?

please tell us when you find one.
guys may add it to http://www.postfix.org/addon.html#logfile


>> ???? ???????:
>> > Hello,
>> >
>> > we use automation to send messages.
>> > from the automation point of view it is nice to know what happened to
>> > message
>> >
>> > I think about the following
>> >
>> > 1) automation send email via smtp --> id of message
>> > 2) automation ask postfix via (rest) api "hey, tell me history of message
>> > id ..."
>> >
>> > any suggestion ?

>сб, 29 сент. 2018 г. в 16:21, Wietse Venema <[hidden email]>:
>> Open a web search engine, ask for for 'logfile analysis tools'.

--
Matus UHLAR - fantomas, [hidden email] ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Despite the cost of living, have you noticed how popular it remains?
Reply | Threaded
Open this post in threaded view
|

Re: any api to read logs ?

Robert Schetterer-2
Am 29.09.2018 um 15:32 schrieb Matus UHLAR - fantomas:

> On 29.09.18 16:42, Илья Шипицин wrote:
>> logfile analys is good for human, it is not rest api.
>
> postfix produces logs, does not read nor parse them.
>
>> I did search already
>>
>> any rest api ?
>
> please tell us when you find one.
> guys may add it to http://www.postfix.org/addon.html#logfile
>
>
>>> ???? ???????:
>>> > Hello,
>>> >
>>> > we use automation to send messages.
>>> > from the automation point of view it is nice to know what happened to
>>> > message
>>> >
>>> > I think about the following
>>> >
>>> > 1) automation send email via smtp --> id of message
>>> > 2) automation ask postfix via (rest) api "hey, tell me history of
>>> message
>>> > id ..."
>>> >
>>> > any suggestion ?
>
>> сб, 29 сент. 2018 г. в 16:21, Wietse Venema <[hidden email]>:
>>> Open a web search engine, ask for for 'logfile analysis tools'.
>

http://sareport.darold.net/
https://www.graylog.org/

and so on

Best Regards
MfG Robert Schetterer

--
[*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64
Schleißheimer Straße 26/MG, 80333 München

Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
Reply | Threaded
Open this post in threaded view
|

Re: any api to read logs ?

Wietse Venema
In reply to this post by Илья Шипицин
Wietse:
> > Open a web search engine, ask for for 'logfile analysis tools'.

???? ???????:
> logfile analys is good for human, it is not rest api.
> I did search already

Here is an idea: combine log analysis with a web API. End of problem.

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: any api to read logs ?

Илья Шипицин


вс, 30 сент. 2018 г. в 4:40, Wietse Venema <[hidden email]>:
Wietse:
> > Open a web search engine, ask for for 'logfile analysis tools'.

???? ???????:
> logfile analys is good for human, it is not rest api.
> I did search already

Here is an idea: combine log analysis with a web API. End of problem.


that was what I was going to implement.
however, I do not like to implement thing that are implemented already.
so, I did a google search and I asked mailing list.

seems, I need to implement rest api myself.


one more question.
I like text logs. they are fast, and available 100% (comparing, for example to some network logging).

is there a way (I did a seach already) to make logs structured ? something like apache access log (field with separator), json, xml, csv ?
whatever to be machine readable (to make rest api actually read it)
 

        Wietse
Reply | Threaded
Open this post in threaded view
|

Re: any api to read logs ?

Tom Hendrikx
Hi,

I have a set of grok patterns for logstash. You can send the postfix
logs to logstash, have them parsed into something more or less
structured by the pattersn, then expose the logstash data through some api.

https://github.com/whyscream/postfix-grok-patterns/

Kind regards,
        Tom


On 01-10-18 07:47, Илья Шипицин wrote:

>
>
> вс, 30 сент. 2018 г. в 4:40, Wietse Venema <[hidden email]
> <mailto:[hidden email]>>:
>
>     Wietse:
>     > > Open a web search engine, ask for for 'logfile analysis tools'.
>
>     ???? ???????:
>     > logfile analys is good for human, it is not rest api.
>     > I did search already
>
>     Here is an idea: combine log analysis with a web API. End of problem.
>
>
>
> that was what I was going to implement.
> however, I do not like to implement thing that are implemented already.
> so, I did a google search and I asked mailing list.
>
> seems, I need to implement rest api myself.
>
>
> one more question.
> I like text logs. they are fast, and available 100% (comparing, for
> example to some network logging).
>
> is there a way (I did a seach already) to make logs structured ?
> something like apache access log (field with separator), json, xml, csv ?
> whatever to be machine readable (to make rest api actually read it)
>  
>
>
>             Wietse
>
Reply | Threaded
Open this post in threaded view
|

Re: any api to read logs ?

Patrick Ben Koetter-2
In reply to this post by Илья Шипицин
* Илья Шипицин <[hidden email]>:

> > Here is an idea: combine log analysis with a web API. End of problem.
>
> that was what I was going to implement.
> however, I do not like to implement thing that are implemented already.
> so, I did a google search and I asked mailing list.
>
> seems, I need to implement rest api myself.
>
>
> one more question.
> I like text logs. they are fast, and available 100% (comparing, for example
> to some network logging).
>
> is there a way (I did a seach already) to make logs structured ? something
> like apache access log (field with separator), json, xml, csv ?
> whatever to be machine readable (to make rest api actually read it)

There isn't and it is one of the (very few) shortcomings of Postfix. AFAIK
logging – as it is today – was hard wired into the code, which means if you
would like to add a new way to output it, i.e. alternate format, structure,
you would have to work your way through all the code.

p@rick

--
[*] sys4 AG
 
https://sys4.de, +49 (89) 30 90 46 64
Schleißheimer Straße 26/MG,80333 München
 
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer, Wolfgang Stief
Aufsichtsratsvorsitzender: Florian Kirstein
 
Reply | Threaded
Open this post in threaded view
|

Re: any api to read logs ?

Илья Шипицин
In reply to this post by Tom Hendrikx


пн, 1 окт. 2018 г. в 12:30, Tom Hendrikx <[hidden email]>:
Hi,

I have a set of grok patterns for logstash. You can send the postfix
logs to logstash, have them parsed into something more or less
structured by the pattersn, then expose the logstash data through some api.

https://github.com/whyscream/postfix-grok-patterns/

nice.
you saved me a lot of time.
 


Kind regards,
        Tom


On 01-10-18 07:47, Илья Шипицин wrote:
>
>
> вс, 30 сент. 2018 г. в 4:40, Wietse Venema <[hidden email]
> <mailto:[hidden email]>>:
>
>     Wietse:
>     > > Open a web search engine, ask for for 'logfile analysis tools'.
>
>     ???? ???????:
>     > logfile analys is good for human, it is not rest api.
>     > I did search already
>
>     Here is an idea: combine log analysis with a web API. End of problem.
>
>
>
> that was what I was going to implement.
> however, I do not like to implement thing that are implemented already.
> so, I did a google search and I asked mailing list.
>
> seems, I need to implement rest api myself.
>
>
> one more question.
> I like text logs. they are fast, and available 100% (comparing, for
> example to some network logging).
>
> is there a way (I did a seach already) to make logs structured ?
> something like apache access log (field with separator), json, xml, csv ?
> whatever to be machine readable (to make rest api actually read it)
>  
>
>
>             Wietse
>
Reply | Threaded
Open this post in threaded view
|

Re: any api to read logs ?

Philip Paeps
In reply to this post by Patrick Ben Koetter-2
On 2018-10-01 09:32:58 (+0200), Patrick Ben Koetter wrote:

> * Илья Шипицин <[hidden email]>:
>>> Here is an idea: combine log analysis with a web API. End of
>>> problem.
>>
>> that was what I was going to implement.
>> however, I do not like to implement thing that are implemented
>> already.
>> so, I did a google search and I asked mailing list.
>>
>> seems, I need to implement rest api myself.
>>
>>
>> one more question.
>> I like text logs. they are fast, and available 100% (comparing, for
>> example
>> to some network logging).
>>
>> is there a way (I did a seach already) to make logs structured ?
>> something
>> like apache access log (field with separator), json, xml, csv ?
>> whatever to be machine readable (to make rest api actually read it)
>
> There isn't and it is one of the (very few) shortcomings of Postfix.
> AFAIK
> logging – as it is today – was hard wired into the code, which
> means if you
> would like to add a new way to output it, i.e. alternate format,
> structure,
> you would have to work your way through all the code.

If you want to go this route, I would recommend looking at libxo[0].  It
will still be a lot of work.

I'm not sure if anyone has ever tried to use libxo for logging though.

Philip

[0] libxo: http://juniper.github.io/libxo/libxo-manual.html

--
Philip Paeps
Senior Reality Engineer
Ministry of Information
Reply | Threaded
Open this post in threaded view
|

Re: any api to read logs ?

Bennett Todd
If I were to depart from the unstructured text syslog format, I'd shift to CSV; with a reasonable choice of quoting rules it can robustly round-trip a lot of tabular data, and very fast writers and parsers are available. I, personally, wouldn't find a more complex structure that easily represents heirarchical structure within a record an improvement. 

On Thu, Oct 4, 2018, 06:06 Philip Paeps <[hidden email]> wrote:
On 2018-10-01 09:32:58 (+0200), Patrick Ben Koetter wrote:

> * Илья Шипицин <[hidden email]>:
>>> Here is an idea: combine log analysis with a web API. End of
>>> problem.
>>
>> that was what I was going to implement.
>> however, I do not like to implement thing that are implemented
>> already.
>> so, I did a google search and I asked mailing list.
>>
>> seems, I need to implement rest api myself.
>>
>>
>> one more question.
>> I like text logs. they are fast, and available 100% (comparing, for
>> example
>> to some network logging).
>>
>> is there a way (I did a seach already) to make logs structured ?
>> something
>> like apache access log (field with separator), json, xml, csv ?
>> whatever to be machine readable (to make rest api actually read it)
>
> There isn't and it is one of the (very few) shortcomings of Postfix.
> AFAIK
> logging – as it is today – was hard wired into the code, which
> means if you
> would like to add a new way to output it, i.e. alternate format,
> structure,
> you would have to work your way through all the code.

If you want to go this route, I would recommend looking at libxo[0].  It
will still be a lot of work.

I'm not sure if anyone has ever tried to use libxo for logging though.

Philip

[0] libxo: http://juniper.github.io/libxo/libxo-manual.html

--
Philip Paeps
Senior Reality Engineer
Ministry of Information