Re: pg_basebackup, pg_receivexlog and data durability (was: silent data loss with ext4 / all current versions)

On 9/12/16 11:16 PM, Michael Paquier wrote:
>> I don't think tar file output in pg_basebackup needs to be fsynced.
>> > It's just a backup file like what pg_dump produces, and we don't fsync
>> > that either. The point of this change is to leave a data directory in
>> > a synced state equivalent to what initdb leaves behind.
> Here I don't agree. The point of the patch is to make sure that what
> gets generated by pg_basebackup is consistent on disk, for all the
> formats. It seems weird to me that we could say that the plain format
> makes things consistent while the tar format may cause some files to
> be lost in case of power outage. The docs make it clear I think:
> + By default, <command>pg_basebackup</command> will wait for all files
> + to be written safely to disk.
> But perhaps this should directly mention that this is done for all the formats?

That doesn't really explain why we fsync. If we think that all
"important" files should be fsynced, why aren't we doing it in pg_dump?
Or psql, or server-side copy? Similarly, there is no straightforward
mechanism by which you can unpack the tar file generated by
pg_basebackup and get the unpacked directory fsynced properly. (I
suppose initdb -S should be recommended.)

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services