Re: Postgres, fsync, and OSs (specifically linux)

Hi,

On 2018-04-28 08:25:53 -0700, Simon Riggs wrote:
> > - Use direct IO. Due to architectural performance issues in PG and the
> > fact that it'd not be applicable for all installations I don't think
> > this is a reasonable fix for the issue presented here. Although it's
> > independently something we should work on. It might be worthwhile to
> > provide a configuration that allows to force DIO to be enabled for WAL
> > even if replication is turned on.
>
> "Use DirectIO" is roughly same suggestion as "don't trust Linux filesystems".

I want to emphasize that this is NOT a linux only issue. It's a problem
across a number of operating systems, including linux.

> It would be a major admission of defeat for us to take that as our
> main route to a solution.

Well, I think we were wrong to not engineer towards DIO. There's just
too many issues with buffered IO to not have a supported path for
DIO. But given that it's unrealistic to do so without major work, and
wouldn't be applicable for all installations (shared_buffer size becomes
critical), I don't think it matters that much for the issue discussed
here.

> The people I've spoken to so far have encouraged us to continue
> working with the filesystem layer, offering encouragement of our
> decision to use filesystems.

There's a lot of people disagreeing with it too.

Greetings,

Andres Freund