Re: Proposal: Adding json logging

Does everyone more or less agree with the following intermediate résumé?

1. Throughout this vivid discussion a good portion of support has already
been manifested for the need of a more structured (machine readable)
logging format. There has been no substantial objection to this need.

2. It has been proposed one JSON object per logging event and alternatively
or complementary one logfmt object per event (although with much less
resonance).

3. Doubts about space have been addressed by the following arguments:
- Data is short lived, as most likely consumed directly without much of a
persistence
- GZIP compression could easily lift space requirements to a near
deduplicacion format's requirements.

4. Doubts about standard conformance are still ongoing discussion, however
there seems not to be in existence yet the ultimate logging standard which
addresses all the shortcomings of existing approaches in an pareto optimal
way. It's most likely something that requires a compromise or options for
the user to choose from.

I hope this is a truthful account of the current state about this thread.

I'd propose, in order to move forward, to wait during a "call for
objections" of a one week period until next Friday if there are any
substantial objections to the direction of this discussion.

Afterwards, I'd like to get, hopefully not alone, some hands dirty.

I hope that's ok for everyone, please let me know if you think this mail is
not a legitimate intention to take a step forward.

Best Regards, David Arnold

El dom., 15 abr. 2018, 10:49 a.m., David Arnold <dar(at)xoe(dot)solutions>
escribió:

> > Exactly - arrays, maps, nested json objects. It's more organized and
> easier to reason about. As postgresql becomes more and more sophisticated
> over time, I see flat logging becoming more unwieldy. With tools like jq,
> reading and querying json on the command line is simple and user friendly,
> and using json for logging capture and aggregation is widely supporting and
> embraced.
>
> Existence and adoption of jq does certainly make a point here over grep
> friendly but less structured data.
>
> But I recommend reading over this blog post https://brandur.org/logfmt
> It's got some strong arguments over a good log format.
> Basically, a good log format is both, human (tail stdaout) and machine
> (log aggregators) readable. Note how logrus solves the conflict.
>
> El dom., 15 abr. 2018, 10:27 a.m., Jordan Deitch <jd(at)rsa(dot)pub> escribió:
>
>> > > I would suggest that the community consider whether postgres will log
>> > multidimensional data. That will weigh into the decision of json vs.
>> > another format quite significantly. I am a fan of the json5 spec (
>> > https://json5.org/), though adoption of this is quite poor.
>> >
>> > What do you mean by multidimensional data? Arrays/maps?
>> >
>> > I think there is no advantage of multidimensional vs prefixed flat
>> logging
>> > unless data structure gets really nastily nested.
>> >
>> > What case where you thinking of?
>>
>> Exactly - arrays, maps, nested json objects. It's more organized and
>> easier to reason about. As postgresql becomes more and more sophisticated
>> over time, I see flat logging becoming more unwieldy. With tools like jq,
>> reading and querying json on the command line is simple and user friendly,
>> and using json for logging capture and aggregation is widely supporting and
>> embraced.
>>
>