Re: Proposal: Adding json logging

On Sat, Apr 14, 2018 at 01:20:16PM -0700, Andres Freund wrote:
> On 2018-04-14 18:05:18 +0200, David Fetter wrote:
> > On Sat, Apr 14, 2018 at 11:51:17AM -0400, Tom Lane wrote:
> > > David Fetter <david(at)fetter(dot)org> writes:
> > > > I think a suite of json_to_* utilities would be a good bit more
> > > > helpful in this regard than changing our human-eye-consumable
> > > > logs. We already have human-eye-consumable logs by default. What
> > > > we don't have, and increasingly do want, is a log format that's
> > > > really easy on machines.
> > >
> > > I'm dubious that JSON is "easier on machines" than CSV.
> >
> > I've found the opposite.
> >
> > CSV is very poorly specified, which makes it at best complicated to
> > build correct parsing libraries. JSON, whatever gripes I have about
> > the format[1] is extremely well specified, and hence has excellent
> > parsing libraries.
>
> Worth to notice that useful json formats for logging also kinda don't
> follow standards. Either you end up with entire logfiles as one big
> array, which most libraries won't parse and makes logrotate etc really
> complicated, or you end up with some easy to parse format where newlines
> have non-standard record separator meaning.

I don't see this as a big problem. The smallest-lift thing is to put
something along the lines of:

When you log as JSON, those logs are JSON objects, one per output
event. They are not guaranteed to break on newlines.

A slightly larger lift would include escaping newlines and ensuring
that JSON output is always single lines, however long.

Best,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate