Re: RFI: Extending the TOAST Pointer

On Wed, May 24, 2023 at 11:50:21AM +0200, Matthias van de Meent wrote:
> I think you might not have picked up on what I was arguing for, but I
> agree with what you just said.
>
> My comment on not needing to invent a new ToastCompressionId was on
> the topic of adding capabilities^ to toast pointers that do things
> differently than the current TOAST and need more info than just sizes,
> 2x 32-bit ID and a compression algorithm.

I am not sure to understand why a new vartag is really required just
for the sake of compression when it comes to VARTAG_EXTERNAL, because
that's where the compression information is stored for ages. The code
footprint gets more invasive, as well if you add more compression
methods as a new vartag implies more code areas to handle.

> ^ capabilities such as compression dictionaries (which would need to
> store a dictionary ID in the pointer), TOAST IDs that are larger than
> 32 bits, and other such advances.

Saying that, I don't really see why we cannot just do both, because it
is clear that many other projects want to fill in more data into
varlena headers for their own needs. Hence, I would do:
1) Use the last bit of va_extinfo in varatt_external to link it more
info related to compression, and keep the compression information
close to varatt_external.
2) Add a new kind of "custom" vartag, for what any other requirements
want it to be.
--
Michael