Re: Proposal: roll pg_stat_statements into core

Greetings,

* Tom Lane (tgl(at)sss(dot)pgh(dot)pa(dot)us) wrote:
> Stephen Frost <sfrost(at)snowman(dot)net> writes:
> > * Alvaro Herrera (alvherre(at)2ndquadrant(dot)com) wrote:
> >> I think there is a false dichotomy here. Migrating an extension out of
> >> contrib doesn't have to equate making it no longer an extension. We
> >> could, instead, keep it being an extension, but move it out of contrib
> >> and into (say) src/extensions/ so that it becomes installed and
> >> available by default, but still an extension. Users won't have to
> >> install a separate contrib package, but they would still have to run
> >> CREATE EXTENSION.
>
> > I don't agree that it's a false dichotomy- from a user experience, you
> > aren't really changing anything with this approach and the entire point
> > is that most of these things should just be available in core.
>
> I don't think that auto-installing such things requires changing much
> of anything. See plpgsql, which is auto-installed now but still sits
> in src/pl/ alongside other PLs that are not auto-installed. Similarly,
> there's nothing really stopping us from choosing to install some
> module of contrib by default; rearranging the source tree is not a
> prerequisite to that.

Sure, I agree with you about all of that- I was just offering it as an
option that if folks felt that rearranging the tree for these
auto-installed things makes sense then, sure, we can do that.

> The situation with pg_stat_statements is more than that, since what
> David is requesting is not merely that we auto-install that extension
> but that we automatically push it into shared_preload_libraries.

Not sure that it'd actually operate in exactly that way, but yes, when
enabled we'd allocate memory the same as if it was in
shared_preload_libraries and such.

> > ...maybe we need a way to turn on/off things like pg_stat_statements but
> > that should have been a runtime "track_stat_statements" or some such,
> > similar to other things like "track_io_timing", not an "independent"
> > extension that is actually developed, managed, and released just as core
> > is.
>
> A key point that hasn't been highlighted in this discussion is that having
> pg_stat_statements as an extension is proof-of-concept that such features
> *can* be implemented outside of core. Don't you think that there are
> probably people maintaining private variants of that extension, who would
> be really sad if we removed or broke APIs they need for it once
> pg_stat_statements is part of core?

Do I feel that out-of-core private extensions are more valuable than the
run-of-the-mill user who just wants to be able to see what queries are
taking time in their running system? Perhaps it's a but cut-throat,
but, uh, no, not in the least. I, at least, don't hack on PostgreSQL
because it's a fantastic platform for closed source/proprietary/third
party solutions to be built upon but rather because it's damn good
open source software that I enjoy working with. That's part of what
frustrates me about contrib- it was once thing X and it's now quite
clearly thing Y but we are refusing to call it that, and that's just not
how I've become accustomed to this open source project working. Let's
call a spade a spade.

That said, I do understand the point that integrating these things into
core *might* lead us down a road where we break extension APIs in ways
that maybe we wouldn't have if they were kept as extensions, but really,
anyone who runs into that issue who really cares to be heard should be
as vocal and active as the PostGIS folks are- we hear from them
regularly on our lists and that's how it should be. I'm not going to be
convinced of the silent-majority argument in this case, given the pain
it causes a great many of our regular users.

... we also have a way to test these hooks/modules in our regression
tests, which we should probably do so that we at least know when we
break them.

Let's put pg_stat_statements aside for a minute- what's the reasoning
behind requireing users to run CREATE EXTENTION to utilize the functions
offered by amcheck? pgstattuple? pg_buffercache? At least with
pg_stat_statements there's good justification, based on performance, for
wanting an on/off switch, but for just about all the other ones, all it
is is a few entries in a catalog table that we're talking about...

Thanks,

Stephen