Re: 9.3: summary of corruption detection / checksums / CRCs discussion

On Tue, Apr 24, 2012 at 3:40 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Sat, Apr 21, 2012 at 7:08 PM, Greg Stark <stark(at)mit(dot)edu> wrote:
>> The earlier consensus was to move all the hint bits to a dedicated
>> area and exclude them from the checksum. I think double-write buffers
>> seem to have become more fashionable but a summary that doesn't
>> describe the former is definitely incomplete.
>
> I don't think we ever had any consensus that moving the hint bits
> around was a good idea.  For one thing, they are only hints in one
> direction.  It's OK to clear them by accident, but it's not OK to set
> them by accident.  For two things, it's not exactly clear how we'd
> rearrange the page to make this work at all: where are those hint bits
> gonna go, if not in the tuple headers?  For three things, index pages
> have hint-type changes that are not single-bit changes.
>
>> That link points to the MVCC-safe truncate patch. I don't follow how
>> optimizations in bulk loads are relevant to wal logging hint bit
>> updates.
>
> That patch actually has more than one optimization in it, I think, but
> the basic idea is that if we could figure out a way to set
> HEAP_XMIN_COMMITTED when loading data into a table created or
> truncated within the same transaction, the need to set hint bits on
> first scan of the table would be eliminated.  Writing the xmin as
> FrozenTransactionId would save even more, though it introduces some
> additional complexity.

This would be great but it's only a corner case. A pretty common
application flow is to write a large number of records, scan them,
update them, scan them again, delete them, etc. in a table that's
already established and possibly pretty large. Unfortunately this
type of work doesn't get a lot of coverage with the common benchmarks.

Also, wouldn't the extra out of band wal traffic from hint bits
exacerbate contention issues on the wal insert lock?

merlin