Re: corrupt pages detected by enabling checksums

On 2013-05-13 16:03:11 +0100, Greg Stark wrote:
> On Mon, May 13, 2013 at 2:49 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> > Sure, the initial file creation will be faster. But are the actual
> > individual wal writes (small, frequently fdatasync()ed) still faster?
> > That's the critical path currently.
> > Whether it is pretty much depends on how the filesystem manages
> > allocated but not initialized blocks...
>
> In ext4 aIui it doesn't actually pick target blocks. It just adjusts
> the accounting so it knows that many blocks will be needed for this
> file and guarantees they'll be available. If you read from them it
> knows to provide 0s. So in theory the performance in the critical path
> would be worse but I think by an insignificant amount.
>
> The reason Postgres pre-allocates the blocks is not for the
> performance optimization. It's for safety. To guarantee -- as best as
> possible -- that it won't get a write error when the time comes to
> write to it. Especially to guarantee that the disk won't suddenly turn
> out to be full.

posix_fallocate() guarantees that. And if you fsync() the file
afterwards its even supposed to still have enough space after the crash.

"After a successful call to posix_fallocate(), subsequent writes to
bytes in the specified range are guaranteed not to fail because of lack
of disk space."

> It seems possible that some file systems would not protect you against
> media errors nearly as well using it.

True. The same probably is true for modifications of existing files for
those fancy COW filesystems...

> I think posix_fallocate is good enough for us and I would support
> using it.

Me too, although this isn't the place where I'd be really excited to see
a patch implementing it properly ;)

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services