Re: Online verification of checksums

Hi,

Am Dienstag, den 18.09.2018, 13:52 -0400 schrieb David Steele:
> On 9/18/18 11:45 AM, Stephen Frost wrote:
> > * Michael Banck (michael(dot)banck(at)credativ(dot)de) wrote:
> > > I have added a retry for this as well now, without a pg_sleep() as well.
> > > This catches around 80% of the half-reads, but a few slip through. At
> > > that point we bail out with exit(1), and the user can try again, which I
> > > think is fine? 
> >
> > No, this is perfectly normal behavior, as is having completely blank
> > pages, now that I think about it. If we get a short read then I'd say
> > we simply check that we got an EOF and, in that case, we just move on.
> >
> > > Alternatively, we could just skip to the next file then and don't make
> > > it count as a checksum failure.
> >
> > No, I wouldn't count it as a checksum failure. We could possibly count
> > it towards the skipped pages, though I'm even on the fence about that.
>
> +1 for it not being a failure. Personally I'd count it as a skipped
> page, since we know the page exists but it can't be verified.
>
> The other option is to wait for the page to stabilize, which doesn't
> seem like it would take very long in most cases -- unless you are doing
> this test from another host with shared storage. Then I would expect to
> see all kinds of interesting torn pages after the last checkpoint.

OK, I'm skipping the block now on first try, as this makes (i) sense and
(ii) simplifies the code (again).

Version 3 is attached.

Michael

--
Michael Banck
Projektleiter / Senior Berater
Tel.: +49 2166 9901-171
Fax: +49 2166 9901-100
Email: michael(dot)banck(at)credativ(dot)de

credativ GmbH, HRB Mönchengladbach 12080
USt-ID-Nummer: DE204566209
Trompeterallee 108, 41189 Mönchengladbach
Geschäftsführung: Dr. Michael Meskes, Jörg Folz, Sascha Heuer

Unser Umgang mit personenbezogenen Daten unterliegt
folgenden Bestimmungen: https://www.credativ.de/datenschutz