Re: Re: Why our counters need to be time-based WAS: WIP: cross column correlation ...

josh(at)agliodbs(dot)com (Josh Berkus) writes:
>> I don't understand what you're talking about at all here. I think
>> there are a lot of unsolved problems in monitoring but the one thing
>> I think everyone is pretty clear on is that the right way to export
>> metrics like these is to export a counter and then have some external
>> component periodically copy the counter into some history table and
>> calculate the derivative, second derivative, running average of the
>> first derivative, etc.
>
> You missed the original point of the discussion, which was to have
> stats we could use for auto-tuning internally. Not to export them.
>
> For example, there are optimizations we could make with the query
> planner if we knew which tables and indexes were "hot" in general.
> That's how we started this discussion, and it's not solved by storing
> the stats history on another server.

There's value to both, and there's no dearth of monitoring frameworks
that people keep on replacing with successors, so there's certainly room
for both ;-).

Recent stuff about such...
<https://lopsa.org/content/philosophy-monitoring>
<https://labs.omniti.com/labs/reconnoiter>

I'm not quite sure what ought to be in PostgreSQL as a "built-in;" I
suspect that what's eventually needed is to be able to correlate things
across database instances, so that when Tom says, "I need to know what
data the planner's working on," the answer can be "OK, got that..."

This data is surely useful to get out of the system, so I'd bias towards
something sorta like what Greg suggests. And the closed-ended answer may
prevent us from asking more sophisticated questions, also not a notably
good thing...
--
(reverse (concatenate 'string "moc.liamg" "@" "enworbbc"))
"If tautologies do not convey information, mathematicians would not be
surprised by them."
-- Mark Miller