Re: Variable length varlena headers redux

Gregory Stark wrote:
>
> Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:
>
> > Gregory Stark <stark(at)enterprisedb(dot)com> writes:
> > > I don't really see a way around it though. Places that fill in VARDATA before
> > > the size (formatting.c seems to be the worst case) will just have to be
> > > changed and it'll be a fairly fragile point.
> >
> > No, we're not going there: it'd break too much code now and it'd be a
> > continuing source of bugs for the foreseeable future. The sane way to
> > design this is that
> >
> > (1) code written to existing practice will always generate 4-byte
> > headers. (Hence, VARDATA() acts the same as now.) That's the format
> > that generally gets passed around in memory.
>
> So then we don't need to replace VARSIZE with SET_VARLENA_LEN at all.
>
> > (2) creation of a short header is handled by the TOAST code just before
> > the tuple goes to disk.
> >
> > (3) replacement of a short header with a 4-byte header is considered
> > part of de-TOASTing.
>
> So (nigh) every tuple will get deformed and reformed once before it goes to
> disk? Currently the toast code doesn't even look at a tuple if it's small
> enough, but in this case we would want it to fire even on very narrow rows.

One weird idea I had was that the macros can read 1 and 4-byte headers,
but can only create 4-byte headers. The code that writes to the shared
buffer pages would to compression from 1 to 4 bytes as needed.

This might avoid changing any macros. It also allows us to carry around
4-byte headers in memory, which I think might be more efficient. I am
not sure if I have heard this idea proposed already or not.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +