pgsql: Refactor rmtree() to use get_dirent_type().

From: Thomas Munro <tmunro(at)postgresql(dot)org>
To: pgsql-committers(at)lists(dot)postgresql(dot)org
Subject: pgsql: Refactor rmtree() to use get_dirent_type().
Date: 2023-01-31 00:49:58
Message-ID: E1pMeqT-000nNW-7u@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Refactor rmtree() to use get_dirent_type().

Switch to get_dirent_type() instead of lstat() while traversing a
directory tree, to see if that fixes the intermittent ENOTEMPTY failures
seen in recent pg_upgrade tests, on Windows CI. While refactoring, also
use AllocateDir() instead of opendir() in the backend, which knows how
to handle descriptor pressure.

Our CI system currently uses Windows Server 2019, a version known not to
have POSIX unlink semantics enabled by default yet, unlike typical
Windows 10 and 11 systems. That might explain why we see this flapping
on CI but (apparently) not in the build farm, though the frequency is
quite low.

The theory is that some directory entry must be in state
STATUS_DELETE_PENDING, which lstat() would report as ENOENT, though
unfortunately we don't know exactly why yet. With this change, rmtree()
will not skip them, and try to unlink (again). Our unlink() wrapper
should either wait a short time for them to go away when some other
process closes the handle, or log a message to tell us the path of the
problem file if not, so we can dig further.

Discussion: https://postgr.es/m/20220919213217.ptqfdlcc5idk5xup%40awork3.anarazel.de

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/54e72b66ed1a55c2fa558bc60d534bdc61dbb62f

Modified Files
--------------
src/common/rmtree.c | 120 ++++++++++++++++++++++++++++------------------------
1 file changed, 64 insertions(+), 56 deletions(-)

Browse pgsql-committers by date

  From Date Subject
Next Message Thomas Munro 2023-01-31 01:00:05 Re: pg_upgrade test failure
Previous Message Tom Lane 2023-01-30 19:13:28 pgsql: Invent "join domains" to replace the below_outer_join hack.