Re: Pluggable toaster

Hi hackers!

Jacob, I agree that the bytea toaster makes a bad example due to core
modification,
and actually is not a good example of an extension.

The vtable concept is intended for less invasive additional functionality -
like providing
detoast iterators in addition to standard detoast mechanics - such
modification requires
only adding iteration methods to toaster and registering them in vtable,
without any
core modifications. I'll add this as a separate commit for generic
(default) Toaster.

It would be more clear for complex data types like JSONB, where developers
could
need some additional functionality to work with internal representation of
data type,
and its full potential is revealed in our JSONB toaster extension. The
JSONB toaster
is still in development but we plan to make it available soon.

For example, we can pass Toaster options with attoptions (I'm currently
working on it)
and these options could, say, allow switching different optimizations in
one toaster like
adding specific compression options or data processing directives, etc.

We doubt that there would be a lot of different custom toasters, because
the Toaster
is quite a complex piece of machinery, but means for extending them would
be heavily
demanded. I have to add some more in-depth explanation of the vtable
concept to
README and the documentation package, the dummy toaster contrib does not
cover
this topic at all.

On installing dummy_toaster contrib: I've just checked it by making a patch
from commit
and applying onto my clone of master and 2 patches provided in previous
email without
any errors and sll checks passed - applying with git am, configure with
debug, cassert,
depend and enable-tap-tests flags and run checks.
Please advice what would cause such a behavior?

Thank you!

On Tue, Sep 13, 2022 at 12:39 AM Jacob Champion <jchampion(at)timescale(dot)com>
wrote:

> On Wed, Aug 24, 2022 at 2:59 AM Nikita Malakhov <hukutoc(at)gmail(dot)com> wrote:
> > I've rebased actual branch onto the latest master and re-created
> patches. Checked with git am,
> > all applied correctly. Please check the attached patches.
> > Rebased branch resides here:
> > https://github.com/postgrespro/postgres/tree/toasterapi_clean
>
> I tried installing and using the dummy_toaster that's provided with
> the gitlink. Upgrade of that cluster fails with the following message:
>
> pg_restore: creating TOASTER "dummy_toaster"
> pg_restore: while PROCESSING TOC:
> pg_restore: from TOC entry 2044; 9861 16390 TOASTER dummy_toaster (no
> owner)
> pg_restore: error: could not execute query: ERROR: unrecognized
> or unsupported class OID: 9861
> Command was: CREATE TOASTER "dummy_toaster" HANDLER
> "public"."dummy_toaster_handler";
>
> I was looking through the thread for a more in-depth description of
> the "vtable" concept, but I didn't see one. It looks like it's just an
> arbitrary extension point, and any new additions would require surgery
> on whatever function needs the particular magic provided by the
> toaster. E.g. your bytea-append toaster extension in the gitlink,
> which still has to modify byteacat() in varlena.c to implement a very
> specific optimization, and then declares its support for that
> hardcoded optimization in the extension.
>
> I'm skeptical that this would remain coherent as it grows. The patch
> claims the vtable API is "powerful", which... I suppose it is, if you
> get to make arbitrary modifications to the core whenever you implement
> it. Did you already have thoughts about which operations would belong
> under that umbrella? What would the procedure be for adding
> functionality to that API? What happens if a toaster wants to
> implement two magic performance optimizations instead of one?
>
> --Jacob
>

--
Regards,
Nikita Malakhov
Postgres Professional
https://postgrespro.ru/