pgsql: Fix concurrency issues with WAL segment recycling on Windows

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: pgsql-committers(at)lists(dot)postgresql(dot)org
Subject: pgsql: Fix concurrency issues with WAL segment recycling on Windows
Date: 2021-03-22 05:03:39
Message-ID: E1lOCj5-0001Cr-MA@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Fix concurrency issues with WAL segment recycling on Windows

This commit is mostly a revert of aaa3aed, that switched the routine
doing the internal renaming of recycled WAL segments to use on Windows a
combination of CreateHardLinkA() plus unlink() instead of rename(). As
reported by several users of Postgres 13, this is causing concurrency
issues when manipulating WAL segments, mostly in the shape of the
following error:
LOG: could not rename file "pg_wal/000000XX000000YY000000ZZ":
Permission denied

This moves back to a logic where a single rename() (well, pgrename() for
Windows) is used. This issue has proved to be hard to hit when I tested
it, facing it only once with an archive_command that was not able to do
its work, so it is environment-sensitive. The reporters of this issue
have been able to confirm that the situation improved once we switched
back to a single rename(). In order to check things, I have provided to
the reporters a patched build based on 13.2 with aaa3aed reverted, to
test if the error goes away, and an unpatched build of 13.2 to test if
the error still showed up (just to make sure that I did not mess up my
build process).

Extra thanks to Fujii Masao for pointing out what looked like the
culprit commit, and to all the reporters for taking the time to test
what I have sent them.

Reported-by: Andrus, Guy Burgess, Yaroslav Pashinsky, Thomas Trenz
Reviewed-by: Tom Lane, Andres Freund
Discussion: https://postgr.es/m/3861ff1e-0923-7838-e826-094cc9bef737@hot.ee
Discussion: https://postgr.es/m/16874-c3eecd319e36a2bf@postgresql.org
Discussion: https://postgr.es/m/095ccf8d-7f58-d928-427c-b17ace23cae6@burgess.co.nz
Discussion: https://postgr.es/m/16927-67c570d968c99567%40postgresql.org
Discussion: https://postgr.es/m/YFBcRbnBiPdGZvfW@paquier.xyz
Backpatch-through: 13

Branch
------
REL_13_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/78c24e97dd189f62187a99ef84016d0eb35a7978

Modified Files
--------------
src/backend/storage/file/fd.c | 22 ++++++++++++++++++----
src/include/pg_config_manual.h | 7 +++++++
2 files changed, 25 insertions(+), 4 deletions(-)

Browse pgsql-committers by date

  From Date Subject
Next Message Robert Haas 2021-03-22 12:47:47 Re: pgsql: Add a new GUC and a reloption to enable inserts in parallel-mode
Previous Message Fujii Masao 2021-03-22 03:05:49 pgsql: pgbench: Improve error-handling in \sleep command.