Re: crash-safe visibility map, take three

On Wed, 2010-12-01 at 11:25 -0500, Robert Haas wrote:
> 1. Every time we observe a page as all-visible, (a) set the
> PD_ALL_VISIBLE bit on the page, without bumping the LSN;

...

> 2. Every time we observe a page as all-visible, (a) set the
> PD_ALL_VISIBLE bit on the page, without bumping the LSN,

My concern here is that both of these proposals introduce something new
that is hint-bit-like, in the sense that we change a data page and
potentially write it to disk without necessarily writing WAL. This will
make it even harder to do CRCs in the future, which are also an
important feature.

I understand the motivation: we don't want to write a page to disk yet
one more time. I think this is only a problem for bulk loading, where it
does seem pretty ridiculous. But for cases other than bulk loading, it
doesn't seem like a problem (likely, there will be other changes to the
page anyway).

Is it worth taking a step back, and saying that we're trying to hard to
use the exact same mechanism for OLTP and bulk loading?

I know bulk loading poses a number of problems. But it might be easier
to make a passable bulk-loading mechanism (not necessarily a general
one) than to continue to fight the WAL-before-data invariant on every
new feature[1].

I'm not proposing anything in specific, just a potential re-framing of
the problem.

Regards,
Jeff Davis

[1] If we obey WAL-before-data everywhere WAL is required, the
correctness can be reasonably understood by mere mortals. But when we
start breaking it for special cases, observe how few people can even
participate in such discussions.