Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On Tue, Apr 17, 2018 at 02:34:53PM -0700, Andres Freund wrote:
> On 2018-04-17 17:29:17 -0400, Bruce Momjian wrote:
> > Also, if we are relying on WAL, we have to make sure WAL is actually
> > safe with fsync, and I am betting only the O_DIRECT methods actually
> > are safe:
> >
> > #wal_sync_method = fsync # the default is the first option
> > # supported by the operating system:
> > # open_datasync
> > --> # fdatasync (default on Linux)
> > --> # fsync
> > --> # fsync_writethrough
> > # open_sync
> >
> > I am betting the marked wal_sync_method methods are not safe since there
> > is time between the write and fsync.
>
> Hm? That's not really the issue though? One issue is that retries are
> not necessarily safe in buffered IO, the other that fsync might not
> report an error if the fd was closed and opened.

Well, we have have been focusing on the delay between backend or
checkpoint writes and checkpoint fsyncs. My point is that we have the
same problem in doing a write, _then_ fsync for the WAL. Yes, the delay
is much shorter, but the issue still exists. I realize that newer Linux
kernels will not have the problem since the file descriptor remains
open, but the problem exists with older/common linux kernels.

> O_DIRECT is only used if wal archiving or streaming isn't used, which
> makes it pretty useless anyway.

Uh, as doesn't 'open_datasync' and 'open_sync' fsync as part of the
write, meaning we can't lose the error report like we can with the
others?

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +