pgsql: Delay fsyncs of pg_basebackup until the end of backup

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: pgsql-committers(at)lists(dot)postgresql(dot)org
Subject: pgsql: Delay fsyncs of pg_basebackup until the end of backup
Date: 2019-09-04 04:26:45
Message-ID: E1i5MsX-0005E2-HS@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Delay fsyncs of pg_basebackup until the end of backup

Since the addition of fsync requests in bc34223 to make base backup data
consistent on disk once pg_basebackup finishes, each tablespace tar file
is individually flushed once completed, with an additional flush of the
parent directory when the base backup finishes. While holding a
connection to the server, a fsync request taking a long time may cause a
failure of the base backup, which is annoying for any integration. A
recent example of breakage can involve tcp_user_timeout, but
wal_sender_timeout can cause similar problems.

While reviewing the code, there was a second issue causing too many
fsync requests to be done for the same WAL data. As recursive fsyncs
are done at the end of the backup for both the plain and tar formats
from the base target directory where everything is written, it is fine
to disable fsyncs when fetching or streaming WAL.

Reported-by: Ryohei Takahashi
Author: Michael Paquier
Reviewed-by: Ryohei Takahashi
Discussion: https://postgr.es/m/OSBPR01MB4550DAE2F8C9502894A45AAB82BE0@OSBPR01MB4550.jpnprd01.prod.outlook.com
Backpatch-through: 10

Branch
------
REL_11_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/996c92b27dfe5e36702f3a5a06c883b76be98705

Modified Files
--------------
src/bin/pg_basebackup/pg_basebackup.c | 24 ++++++++++++++----------
1 file changed, 14 insertions(+), 10 deletions(-)

Browse pgsql-committers by date

  From Date Subject
Next Message Michael Paquier 2019-09-04 06:47:59 pgsql: Fix thinko when ending progress report for a backend
Previous Message Alvaro Herrera 2019-09-03 21:42:16 pgsql: Make XLogReaderInvalReadState static