Re: [RFC] indirect toast tuple support

On Tue, Feb 19, 2013 at 9:26 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> On 2013-02-19 09:12:02 -0500, Robert Haas wrote:
>> On Tue, Feb 19, 2013 at 9:00 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
>> >> So the other way that we could do this is to use something that's the
>> >> same size as a TOAST pointer but has different content - the
>> >> seemingly-obvious choice being va_toastrelid == 0.
>> >
>> > Unfortunately that would mean you need to copy the varatt_external (or
>> > whatever it would be called) to aligned storage to check what it
>> > is. Thats why I went the other way.
>>
>> How big a problem is that, though?
>
> There are quite some places where we test the actual type of a Datum
> inside tuptoaster.c. Copying it to local storage everytime might
> actually be noticeable performancewise. Besides the ugliness of needing
> a local variable, copying the data and only testing afterwards...

Hrm, OK.

>> >> I'd be a little
>> >> reluctant to do it the way you propose because we might, at some
>> >> point, want to try to reduce the size of toast pointers. If you have
>> >> a tuple with many attributes, the size of the TOAST pointers
>> >> themselves starts to add up. It would be nice to be able to have 8
>> >> byte or even 4 byte toast pointers to handle those situations. If we
>> >> steal one or both of those lengths to mean "the data is cached in
>> >> memory somewhere" then we can't use those lengths in a smaller on-disk
>> >> representation, which would seem a shame.
>> >
>> > I agree. As I said above, having the type overlayed into the lenght was
>> > and is a bad idea, I just haven't found a better one thats compatible
>> > yet.
>> > Except inventing typlen=-3 aka "toast2" or something. But even that
>> > wouldn't help getting rid of existing pg_upgraded tables. Besides being
>> > a maintenance nightmare.
>> >
>> > The only reasonable thing I can see us doing is renaming
>> > varattrib_1b_e.va_len_1be into va_type and redefine VARSIZE_1B_E into a
>> > switch that maps types into lengths. But I think I would put this off,
>> > except placing a comment somewhere, until its gets necessary.
>>
>> I guess I wonder how hard we think it would be to insert such a thing
>> when it becomes necessary. How much stuff is there out there that
>> cares about the fact that that length is a byte?
>
> You mean whether we could store the length in 6 bytes and use two for
> the type? That should probably work as well. But I don't see much
> advantage in that given that all those sizes ought to be static.
> Redefining VARSIZE_1B_E as indicated above should be fairly easy, there
> aren't many callsites that touch stuff at such low level.

/me blinks.

No, that's not what I meant. I meant: how hard it would be to
redefine VARSIZE_1B_E along the lines you suggest?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company