LONG varsize - how to go on

How I would like to go on:

What I've done so far is to prepare the HeapTuple handling so
it should be possible to reallocate the t_data part from
inside the heap_insert() and heap_update().

Next would be to define how the runtime configuration is to
be stored and configured. So I can get at least the minimum
required system catalog changes made soon. If we leave out
auto-compression for now, a rellongrelid (type Oid default
Invalid) in pg_class, and a attcanlong (type bool default
true) in pg_attribute would do it. This would give space to
enable a single relation for this feature, and then disable
single columns again. New utility commands will finally gain
access to the features. Some scripts will do it during
development, so the feature will not show up - thus beeing
silently unavailable to the user - until we want to ship it.

I think these are the best places to put the configuration
into, because the information would be already available at
no cost inside heap access (rel->rd_rel->rellongrelid and
rel->rd_att...).

What I want to implement for now, is to store extended
VARLENA attributes into a regular (but other relkind)
relation with the Oid key as discussed. That will cause
minimum changes to VACUUM. If storage/retrieval of attributes
is encapsulated right, it could get replaced by something
different at any time in the future when we have enough
experience with this new technique.

Christof Petig and me then could start implementing it, using
lztext with locally disabled compression feature for the
basics. I'll not commit any changes until after feature
freeze of the upcoming release. During the last changes (for
HeapTuple handling) I've seen many places in the code, that
deal themself on VARSIZE/VARDATA with text type attributes,
which then must use textout() instead (what IMHO is better
anyway). So implementing an ALL-varsize move off, instead of
a special LONG datatype, will take longer than February 1st.

This plan means in summary:

1. Full possible configurability with minimum catalog
changes.

2. Hidden without any side effects until we agree to enable
it.

3. Later optimizable storage of extended attributes.

I can't see any reason to way much longer. Can we please have
a consensus to get started?

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck(at)debis(dot)com (Jan Wieck) #