Re: Block-level CRC checks

On Thu, 2008-10-02 at 16:18 -0400, Alvaro Herrera wrote:

> > > Maybe we could mix this with Simon's approach to counting hint bit
> > > setting, and calculate a valid CRC on the page every n-th non-logged
> > > change.
> >
> > I still think we should only calculate checksums on the actual write.
>
> Well, if we could trade off a bit of performance for correctness, I
> would give up on that :-) However, you're right that this tradeoff is
> not what we're having here.
>
> > And, this still seems to have an issue with WAL, unless Simon's
> > original idea somehow included recording hint bit settings/dirtying
> > the page in WAL.
>
> I have to admit I don't remember exactly how it worked :-) I think the
> idea was avoiding setting the page dirty until a certain number of hint
> bit setting operations had been done (which I think means it's not
> useful for the present purpose).

Having read everything so far, the only way I can see to solve the
problem does seem to be to make hint bit setting write WAL. When, is the
question. Every time is definitely the wrong answer.

Hint bit setting approach so far is in two parts: we add code to
separate the concept of "dirty" from "has hint bits set". We already
have some logic that says when to write dirty pages. So we just add some
slightly different logic that says when to write hinted pages.

The main correctness of the idea has been worked out. The difficult part
is the "when to write hinted pages" because its just a heuristic,
subject to argument and lots of performance testing.

--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support