Re: Pluggable toaster

Hi Nikita,

> Setting TOAST for table and database is a subject for discussion. There is already default
> Toaster. Also, there is not much sense in setting Jsonb Toaster as default even for table, do
> not say database, because table could contain other TOASTable columns not of Json type.

True, but besides Jsonb Toaster one can implement a universal one as
well (your own example: encryption, or a Toaster that bypasses a 1 GB
value limitation). However we can probably keep this out of scope of
this particular patchset for now. As mentioned before this is going to
be just an additional syntax for the user convenience.

> Custom Toasters will work with Extended storage, but as I answered in previous email -
> there is no much use of it, because they would deal with compressed data.

Compression is actually a part of the four-stage TOAST algorithm. So
what you're doing is using the default TOAST most of the time, but if
the storage strategy is EXTERNAL and a custom TOASTer is specified
then you use a type-aware "TOASTer".

If the goal is to implement true "Pluggable TOASTers" then the TOAST
should be substituted entirely. If the goal is to implement type-aware
sub-TOASTers we should be honest about this and call it otherwise:
e.g. "Type-aware TOASTer" or perhaps "subTOASTer". Additionally in
this case there should be no validate() method since this is going to
work only with the default heapam that implements the default TOASTer.

So to clarify, the goal is to deliver true "Pluggable TOASTers" or
only "type-aware sub-TOASTers"?

> For TOASTer you SHOULD distinguish insert and update operations, really. Because for
> TOASTed data these operations affect many tuples, and AM does know which of them
> were updated and which were not - that's very serious limitation of current TOAST, and
> TOAST mechanics shoud care about updating affected tuples only instead of marking
> whole record dead and inserting new one. This is also an argument for not using EXTENDED
> storage mode - because with compressed data you do not have such choice, you should
> drop the whole record.

This may actually be a good point. I suggest reflecting it in the documentation.

> >Users should be able to DROP extension. I seriously doubt that the
> >patch is going to be accepted as long as it has this limitation.
>
> There is a mention in documentation and previous discussion that this operation would lead
> to loss of data TOASTed with this custom Toaster.

I don't see any reason why the semantics for Toasters should be any
different from user-defined types implemented in an extension. If
there are columns that use a given Toaster we should forbid DROPping
the extension. Otherwise "DROP extension" should succeed. No one says
that this should be a fast operation but it should be possible.

[1]: https://www.postgresql.org/message-id/flat/CAJ7c6TOtAB0z1UrksvGTStNE-herK-43bj22=5xVBg7S4vr5rQ(at)mail(dot)gmail(dot)com

--
Best regards,
Aleksander Alekseev