Re: Maliing list request: pgsql-forks@

On Wed, 31 Jul 2019 at 08:43, Nasby, Jim <nasbyj(at)amazon(dot)com> wrote:

>
> In case it adds some better perspective, here’s the list of pain points
> that were captured in the BoF:
>
>
For what it's worth, though I entirely agree with the below I've also been
truly amazed at how much can be achieved with PostgreSQL extensions.

Even in some Petr has been able to do things with BDR has been able to do
things I just didn't think would be possible, let alone practical, and
often done them quite cleanly.

> Backend static functions / structs
>

Yeah. I've had to copy postgres code too many times because the
functionality is in core exactly where I need it, but I can't access it.

Sometimes I understand the deliberate hiding of access, but often it's
pretty arbitrary.

A bigger symbol table is not really much concern for C code. It's not like
we're working with C++ where the symbol table is many many megabytes of
template mangled gibberish that must be trimmed at all costs to avoid
giving the dynamic linker a heart attack.

We don't have any formal API or any well defined approach to deciding what
is/isn't "public" so it's mostly pretty arbitrary what is visible and what
is not.

Not enough hooks
>

Never enough hooks. ALL. THE. HOOKS.

We should really get together, make a list of what we want, and start
working together to add them. They're cheap and there are a variety of
tricks available to make them cheaper still if we need them in hot paths.

I looked around a while ago for a well supported, portable runtime patching
library to support NOOP jump replacement based hooking btw. Unfortunately I
didn't find anything that looks remotely viable, let alone usable by
something that needs a high level of of maturity and portability like
PostgreSQL.

> Ordering of hook registration
>

Haven't really had to face this one. I'm interested?

> More pluggable modules/components
>

Like? Not arguing, but anything like that can have a big cost in
maintenance on core and the interface design is often hard.

> Extension upgrading is difficult
>

Couldn't agree more. Extension upgrades are ok-ish when linear, but as soon
as you start doing stable/dev versions and branching it's hell.

Hard to extend grammar (add more WITH options?)
>

It'd be nice to have a few limited extension points like that, but the
discussions around more general parser extension have tended to flounder
for a variety of good reasons. Our parser tooling is extremely poorly
suited to pluggable extensions, and that aside it's really hard to do
things like maintain good clear error messages while making the grammar
extensible in general terms.

Problems with extensions calling other extensions (both SQL and C; this
> item is actually broader than the others)
>

Really?

We do this tons in 2ndQuadrant, to the point where BDR3 is an extension on
top of pglogical3. It works very well and we've had hardly any issues.

The most crucial thing is using rendezvous variables - see
find_rendezvous_variable in fmgr.h. These are woefully under-documented and
at some point I'd love to change that because I concocted a number of
schemes ranging from awful to merely ugly to help extensions find each
others' API exports before Petr spotted rendezvous variables.

The PostgreSQL extension docs could definitely use some additions to cover
bgworkers, discussion of DirectFunctionCallN, caching fmgrinfo, etc too.
One day...

Still too much is_superuser() (possibly also broader)
>

I almost wonder if we should be able to hook is_superuser, but I doubt it'd
be much use without having a way to determine the calling context. Better
to just fix the is_superuser() stuff to use proper privileges.

--
Craig Ringer http://www.2ndQuadrant.com/
2ndQuadrant - PostgreSQL Solutions for the Enterprise