Performance monitoring (was: [PATCHES] Logging checkpoints and other slowdown causes)

Moving to -hackers.

On Fri, May 11, 2007 at 04:37:44PM +0100, Heikki Linnakangas wrote:
> >If you know when the checkpoint ended, and you know how long each of the
> >pieces took, you can reconstruct the other times easily. The way you
> >describe this it is true--that the summary is redundant given the
> >detail--but if you put yourself in the shoes of a log file parser the
> >other way around is easier to work with. Piecing together log entries
> >is a pain, splitting them is easy.
> >
> >If I had to only keep one line out of this, it would be the one with the
> >summary. It would be nice to have it logged at INFO.
>
> Yeah, if we have the summary line we don't need the other lines and vice
> versa. I have sympathy for parsing log files, I've done that a lot in
> the past and I can see what you mean. Having the individual lines is
> nice when you're monitoring a running system; you don't get the summary
> line until the checkpoint is finished. I suppose we can have both the
> individual lines and the summary, the extra lines shouldn't hurt anyone,
> and you won't get them unless you turn on the new log_checkpoints
> parameter anyway.

Not to beat a dead horse, but do we really want to force folks to be
parsing logs for performance monitoring? Especially if that log parsing
is just going to result in data being inserted into a table anyway?

I know there's concern about performance of the stats system and maybe
that needs to be addressed, but pushing users to log parsing is a lot of
extra effort, non-standard, likely to be overlooked, and doesn't play
well with other tools. It also conflicts with all the existing
statistics framework.
--
Jim Nasby jim(at)nasby(dot)net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)