Re: [PATCH] Atomic pgrename on Windows

On Tue, Jan 7, 2020 at 11:04 PM Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Alexander Korotkov <a(dot)korotkov(at)postgrespro(dot)ru> writes:
> > I'm not sure issue we faced is really about single platform. TBH, the
> > assumptions we place to rename function is very strict. We assume
> > rename works atomically on system crash. And we indirectly assume it
> > can work concurrently as least with file readers. The both are
> > probably true for Linux with most popular filesystems. But we do
> > support pretty many platforms. I think the issue we didn't
> > investigate rename properties well on all of them. But if we do, it
> > might happen some assumptions are wrong on multiple platforms.
> > Windows is just used busy enough to spot the problem.
>
> Well, atomic rename is required by POSIX. True, we have found bugs
> related to that in one or two Unix-ish platforms. But nobody
> is going to deny that those are OS bugs that the OS ought to fix,
> rather than accepted behaviors that applications have to find
> some way to work around. I'm not pleased with the idea that
> Windows' deficiencies in this area should result in kluges all over
> our code. I think we should stick to the autoconf recommendation:
>
> Autoconf follows a philosophy that was formed over the years by
> those who have struggled for portability: isolate the portability
> issues in specific files, and then program as if you were in a
> Posix environment.

I get this point: we want to program postgres like it's working in
perfect POSIX environment.

And POSIX standard really requires rename() to work concurrently with
file accesses.

"If the link named by the new argument exists and the file's link
count becomes 0 when it is removed and no process has the file open,
the space occupied by the file shall be freed and the file shall no
longer be accessible. If one or more processes have the file open when
the last link is removed, the link shall be removed before rename()
returns, but the removal of the file contents shall be postponed until
all references to the file are closed."

But issue is that on Windows POSIX rename() is kind of impossible to
implement. And I suspect other platforms may have issues too.

Regarding "pg_stat_tmp/global.stat", which is a problem in particular
case, we may evade file renaming altogether. Instead, we may
implement shared-memory counter for filename. So, instead of
renaming, new reads will just come to new file.

------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company