Re: Block-level CRC checks

Gregory Stark wrote:

> I think you could checksum the block including the hint bits then go back and
> remove them from the checksum.

I'm not sure what you're proposing here. It sounds to me like you are
saying that we can read the page, make it available to other users, and
then check the CRC. I don't think this works though, because if you do
that the possibly-invalid buffer is available to the other readers.

> I didn't realize you were handling more than just the heap transaction
> hint bits though. It would be hard to do it in any kind of abstract
> away like you were describing.

Yeah, I also initially thought that there was only a single set of hint
bits, but that turned out not to be the case. Right now the nbtree hint
bits are the ones missing :-( It's hard to see how to handle those.

> How happy are you with the wal logging entries? Have you done any tests to see
> how much extra wal traffic it is? Are you sure you always generate enough logs
> soon enough?

I haven't measured the amount of traffic. They are always generated
"soon enough": just before calling smgrwrite on the page on FlushBuffer,
i.e. just before the page hits disk. I admit it feels a bit dirty to be
calling XLogInsert on such low a level.

Right now we log all bits for all tuples, even if a single bit changed.
It could be more efficient if I could only logs tuples whose hints bits
had changed since the last write. This would require setting a bit on
every tuple "this tuple has an unlogged hint bit" (right now there's a
bit at the page level). I haven't tried implementing that.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support