Re: Block-level CRC checks

On Tue, Dec 1, 2009 at 10:47 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Bruce Momjian <bruce(at)momjian(dot)us> writes:
>> Greg Stark wrote:
>>> It should be relatively cheap to skip the hint bits in the line
>>> pointers since they'll be the same bits of every 16-bit value for a
>>> whole range. Alternatively we could just CRC the tuples and assume a
>>> corrupted line pointer will show itself quickly. That would actually
>>> make it faster than a straight CRC of the whole block -- making
>>> lemonade out of lemons as it were.
>
> I don't think "relatively cheap" is the right criterion here --- the
> question to me is how many assumptions are you making in order to
> compute the page's CRC.  Each assumption degrades the reliability
> of the check, not to mention creating another maintenance hazard.

Well the only assumption here is that we know where the line pointers
start and end. That sounds like the same level of assumption as your
structure with the line pointers moving around. I agree with your
general point though -- trying to skip the hint bits strewn around in
the tuples means that every line pointer had better be correct or
you'll be in trouble before you even get to the CRC check. Skipping
them in the line pointers just means applying a hard-coded mask
against each word in that region.

It seems to me adding a third structure on the page and then requiring
tqual to be able to find that doesn't significantly reduce the
complexity over having tqual be able to find the line pointers. And it
significantly increases the complexity of every other part of the
system which has to deal with a third structure on the page. And
adding and compacting the page becomes a lot more complex. I'm also
I'm a bit leery about adding more line pointers than necessary because
even a small number of line pointers will mean you're likely to often
fit one fewer tuple on the page.

--
greg