Re: [HACKERS] LONG varsize - how to go on

> How I would like to go on:
>
> What I've done so far is to prepare the HeapTuple handling so
> it should be possible to reallocate the t_data part from
> inside the heap_insert() and heap_update().

OK, I assume this is so you can change the tuple size on the fly when
inserting the tuple.

>
> Next would be to define how the runtime configuration is to
> be stored and configured. So I can get at least the minimum
> required system catalog changes made soon. If we leave out
> auto-compression for now, a rellongrelid (type Oid default
> Invalid) in pg_class, and a attcanlong (type bool default
> true) in pg_attribute would do it. This would give space to
> enable a single relation for this feature, and then disable
> single columns again. New utility commands will finally gain
> access to the features. Some scripts will do it during
> development, so the feature will not show up - thus beeing
> silently unavailable to the user - until we want to ship it.

Got it. You want to store the oid of the long table for the base table
inside the pg_class/Relation structure. Good idea.

>
> I think these are the best places to put the configuration
> into, because the information would be already available at
> no cost inside heap access (rel->rd_rel->rellongrelid and
> rel->rd_att...).
>

Yes, I recommand a macro so access is clear. See
RelationGetRelationName for an example.

> What I want to implement for now, is to store extended
> VARLENA attributes into a regular (but other relkind)
> relation with the Oid key as discussed. That will cause
> minimum changes to VACUUM. If storage/retrieval of attributes
> is encapsulated right, it could get replaced by something
> different at any time in the future when we have enough
> experience with this new technique.

Good. I recommend calling it pg_* so it is automatically excluded from
client table lists.

>
> Christof Petig and me then could start implementing it, using
> lztext with locally disabled compression feature for the

I would recommand having compression disabled by default, and enabled by
the user.

> basics. I'll not commit any changes until after feature
> freeze of the upcoming release. During the last changes (for
> HeapTuple handling) I've seen many places in the code, that
> deal themself on VARSIZE/VARDATA with text type attributes,
> which then must use textout() instead (what IMHO is better
> anyway). So implementing an ALL-varsize move off, instead of
> a special LONG datatype, will take longer than February 1st.

OK, so we are going to see this after 7.0 is released, which is fine. I
understand the concern about all the accesses to VARDATA() inside the
code. That will clearly be difficult.

>
> This plan means in summary:
>
> 1. Full possible configurability with minimum catalog
> changes.
>
> 2. Hidden without any side effects until we agree to enable
> it.
>
> 3. Later optimizable storage of extended attributes.
>
> I can't see any reason to way much longer. Can we please have
> a consensus to get started?

Sounds good.

--
Bruce Momjian | http://www.op.net/~candle
maillist(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026