Re: RFI: Extending the TOAST Pointer

On Thu, May 18, 2023 at 8:06 AM Matthias van de Meent
<boekewurm+postgres(at)gmail(dot)com> wrote:
> This enum still has many options to go before it exceeds the maximum
> of the uint8 va_tag field. Therefore, I don't think we have no disk
> representations left, nor do I think we'll need to add another option
> to the ToastCompressionId enum.
> As an example, we can add another VARTAG option for dictionary-enabled
> external toast; like what the pluggable toast patch worked on. I think
> we can salvage some ideas from that patch, even if the main idea got
> stuck.

Adding another VARTAG option is somewhat different from adding another
ToastCompressionId. I think that right now we have embedded in various
places the idea that VARTAG_EXTERNAL is the only thing that shows up
on disk, and we'd need to track down all such places and adjust them
if we add other VARTAG types in the future. Depending on how it is to
be used, adding a new ToastCompressionId might be less work. However,
I don't think we can use the possibility of adding a new VARTAG value
as a reason why it's OK to use up the last possible ToastCompressionId
value for something non-extensible.

For projects like this, the details matter a lot. If the goal is to
add a new compression type that behaves like the existing compression
types, more or less, then I think we should allocate the last
ToastCompressionId bit to mean "some other compression ID" and add a
1-byte header that says which one is in use. But if the new feature
being added is enough different from the other compression methods,
then it might be better to do it in some other way e.g. a new VARTAG.

--
Robert Haas
EDB: http://www.enterprisedb.com