Re: Production block comparison facility

On Tue, Jul 29, 2014 at 7:30 PM, Heikki Linnakangas
<hlinnakangas(at)vmware(dot)com> wrote:
> I don't understand how this works. A full-page image contains the new page
> contents *after* the WAL-logged operation. For example, in a heap insert,
> the full-page image contains the new tuple. How can you compare that with
> what's on the disk already?

An exact match of the FPW and the current page is not done, the patch
as it stands now checks if a FPW is consistent with the content of
current page by checking if it does not include changes that diverge
from what the FPW has.
For example for a heap insert, if current page has N records
pointer1/tup1..pointerN/tupN, FPW should only contain (N+1) records
pointer1/tup1..pointer(N+1)/tup(N+1). After applying the mask at block
recovery, process simply checks that the FPW and current page contain
the first N records, marking FPW and current page as inconsistent if
the current page has some garbage like some extra tuple entries not in
the FPW. I am sure you have arguments against that though...

> ISTM you'd need to log two full-page images for every WAL record. A before
> image and an after image.
The after image is the current FPW, so there is nothing else to do for
it. But for the before buffer, what do you think about using
ReadBufferExtended with RBM_NORMAL? We could grab its content from
disk in XLogInsert only when we are sure that a backup block is added.

> Then you could do a lot of checking:
> 1. the before image should match what's on disk already
> 2. the result after applying the WAL record should match the after image.
A WAL record can contain up to XLR_MAX_BKP_BLOCKS backup blocks.
Should we double it from 4 to 8?

> That would be more handy than the approach I used, where the page images are
> logged to a separate file. You wouldn't need to deal with any new files, as
> all the data is in the WAL. Verification would be done directly in the
> standby, with no need to run any extra programs.
In this case, would it better to control that with a GUC? Making that
the default will increase the amount of WAL for all types of
applications, except if couple with FPW compression...
Regards,
--
Michael