Re: Better support for whole-row operations and composite types

Joe Conway <mail(at)joeconway(dot)com> writes:
> Tom Lane wrote:
>> We will be able to make generic I/O routines for composite types,
>> comparable to those used now for arrays. Not sure what a convenient
>> external format would look like. (Possibly use the same conventions
>> as for a 1-D array?)

> So you mean like an array, but with possibly mixed datatypes?
> '{1 , "abc def", 2.3}'
> Seems to make sense.

The unresolved question in my mind is how to represent NULL elements.
However, we have to solve that sooner or later for arrays too. Any
thoughts?

> Another option might be to use the ROW keyword, something like:
> ROW[1 , 'abc', 2.3]

This is a separate issue, just as the ARRAY[] constructor has different
uses from the array I/O representation. I do want some kind of runtime
constructor, but ROW[...] doesn't get the job done because it doesn't
provide any place to specify the rowtype name. Maybe we could combine
ROW[...] with some sort of cast notation?

ROW[1 , 'abc', 2.3] :: composite_type_name
CAST(ROW[1 , 'abc', 2.3] AS composite_type_name)

Does SQL99 provide any guidance here?

>> TupleDescGetSlot: no-op, returns NULL TupleGetDatum: ignore slot,
>> return tuple t_data pointer as datum
>>
>> This will work because heap_formtuple and BuildTupleFromCStrings can
>> return a HeapTuple whose t_data part is already a valid row Datum,
>> simply by setting the appropriate length and type fields in it. (If
>> the tuple is ever stored to disk as a regular table row, these fields
>> will be overwritten with xmin/cmin info at that time.)

> Is this the way you did things in your recent commit?

Almost. I ended up keeping TupleDescGetSlot as a live function, but its
true purpose is only to ensure that the tupledesc gets registered with
the type cache (see BlessTupleDesc() in CVS tip). The slot per se never
gets used. I believe that CVS tip is source-code-compatible with
existing SRFs, even though I adjusted all the ones in the distribution
to stop using the TupleTableSlot stuff.

The main point though is that row Datums now contain sufficient info
embedded in them to allow runtime type lookup the same as we do for arrays.

>> To convert a row Datum into something that can be passed to
>> heap_getattr, one could use a local variable of type HeapTupleData
>> and set its t_data field to the datum's pointer value. t_len is
>> copied from the datum contents, while the other fields of
>> HeapTupleData can just be set to zeroes.

> I think I understand this, but an example would help.

There are several in the PL sources now, for instance plpgsql does this
with an incoming rowtype argument:

if (!fcinfo->argnull[i])
{
HeapTupleHeader td;
Oid tupType;
int32 tupTypmod;
TupleDesc tupdesc;
HeapTupleData tmptup;

td = DatumGetHeapTupleHeader(fcinfo->arg[i]);

/* Extract rowtype info and find a tupdesc */
tupType = HeapTupleHeaderGetTypeId(td);
tupTypmod = HeapTupleHeaderGetTypMod(td);
tupdesc = lookup_rowtype_tupdesc(tupType, tupTypmod);

/* Build a temporary HeapTuple control structure */
tmptup.t_len = HeapTupleHeaderGetDatumLength(td);
ItemPointerSetInvalid(&(tmptup.t_self));
tmptup.t_tableOid = InvalidOid;
tmptup.t_data = td;

exec_move_row(&estate, NULL, row, &tmptup, tupdesc);
}

This is okay because the HeapTupleData is not needed after the call to
exec_move_row.

>> * We have to be able to re-use an already-existing cache entry if it
>> matches a requested TupleDesc.

> For anonymous record types, how will that lookup be done efficiently?
> Can the hash key be an array of attribute oids?

Right, that's the way I did it. See src/backend/utils/cache/typcache.c

> As an aside, it would be quite useful to have support for arrays of
> tuples. Any idea on how to do that without needing to define an explicit
> array type for each tuple type?

Hmm, messy ...

I wonder now whether we still really need a separate pg_type entry for
every array type. The original motivation for doing that has been at
least partly subsumed by storing element type OIDs inside the arrays
themselves. I wonder if we could go over to a scheme where, say,
atttypid is the base type ID and attndims being nonzero is what you
check to find out it's really an array of atttypid. Not sure how we
could map that idea into function and expression args/results, though.

Plan B would be to go ahead and create array types. Not sure I would
want to do this for table rowtypes, but if we did it only for CREATE
TYPE AS then it doesn't sound like an unreasonable amount of overhead.

regards, tom lane