Re: Pluggable cumulative statistics

On Thu, Jul 04, 2024 at 10:11:02AM +0700, Andrei Lepikhov wrote:
> It is a feature my extensions (which usually change planning behaviour)
> definitely need. It is a problem to show the user if the extension does
> something or not because TPS smooths the execution time of a single query
> and performance cliffs.

Yeah, I can get that. pgstat.c is quite good regarding that as it
delays stats flushes until commit by holding pending entries (see the
pgStatPending business for variable-size stats). Custom stats kinds
registered would just rely on these facilities, including snapshot
APIs, etc.

> BTW, we have 'labelled DSM segments', which allowed extensions to be
> 'lightweight' - not necessarily be loaded on startup, stay backend-local and
> utilise shared resources. It was a tremendous win for me.
>
> Is it possible to design this extension in the same way?

I am not sure how this would be useful when it comes to cumulative
statistics, TBH. These stats are global by design, and especially
since these most likely need to be flushed at shutdown (as of HEAD)
and read at startup, the simplest way to achieve that to let the
checkpointer and the startup process know about them is to restrict
the registration of custom stats types via _PG_init when loading
shared libraries. That's what we do for custom WAL RMGRs, for
example.

I would not be against a new flag in KindInfo to state that a given
stats type should not be flushed, as much as a set of callbacks that
offers the possibility to redirect some stats kinds to somewhere else
than pgstat.stat, like pg_stat_statements. That would be a separate
patch than what's proposed here.
--
Michael