Re: [RFC] indirect toast tuple support

On Sat, Feb 16, 2013 at 4:42 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> I propose extending the EXTERNAL varlenas to be able to point to memory
> instead just to disk. It seem apt to use EXTERNAL for this as they
> aren't stored in the normal heap tuple but somewhere else.
> Unfortunately there is no backward-compatible flag space in
> varattrib_1b_e either to nicely denote this and we sure don't want to
> break on-disk compatibility for this. Thus I propose to distinguish
> on-disk and in-memory tuples via the varattrib_1b_e.va_len_1be.

The intention was to use va_len_1be to allow extensibility with
different external toast types. We were originally not going to have
it at all and just before committing we became concerned that we
wanted to avoid painting ourselves into a corner where we wouldn't be
able to come up with any new formats for external toast types. So we
added this one byte. i suggest thinking of it more as a "type" field
that happens to be the length of the toast pointer by convention than
an actual length header.

Just as an example I have at various times proposed a column
compression method that would store a dictionary of common values and
the toast pointer would be a pointer to this dictionary and an index
into it.

I have no problem using it for this case since it's using up only one
particular value for this field. As long as you don't want to use up
all the possible values for a single type of external pointer it seems
in line with the plan.

--
greg