Re: Checksums by default?

* Andres Freund (andres(at)anarazel(dot)de) wrote:
> On 2017-01-25 19:30:08 -0500, Stephen Frost wrote:
> > * Peter Geoghegan (pg(at)heroku(dot)com) wrote:
> > > On Wed, Jan 25, 2017 at 3:30 PM, Stephen Frost <sfrost(at)snowman(dot)net> wrote:
> > > > As it is, there are backup solutions which *do* check the checksum when
> > > > backing up PG. This is no longer, thankfully, some hypothetical thing,
> > > > but something which really exists and will hopefully keep users from
> > > > losing data.
> > >
> > > Wouldn't that have issues with torn pages?
> >
> > No, why would it? The page has either been written out by PG to the OS,
> > in which case the backup s/w will see the new page, or it hasn't been.
>
> Uh. Writes aren't atomic on that granularity. That means you very well
> *can* see a torn page (in linux you can e.g. on 4KB os page boundaries
> of a 8KB postgres page). Just read a page while it's being written out.
>
> You simply can't reliably verify checksums without replaying WAL (or
> creating a manual version of replay, as in checking the WAL for a FPW).

Looking through the WAL isn't any surprise and is something we've been
planning to do for other reasons anyway.

Thanks!

Stephen