Re: Postgresql13_beta1 (could not rename temporary statistics file) Windows 64bits

Em ter., 16 de jun. de 2020 às 01:10, Justin Pryzby <pryzby(at)telsasoft(dot)com>
escreveu:

> On Mon, Jun 15, 2020 at 11:49:33PM -0300, Ranier Vilela wrote:
> > I can confirm that the problem is in pgrename (dirmod.c),
> > something is not OK, with MoveFileEx, even with the
> > (MOVEFILE_REPLACE_EXISTING) flag.
> >
> > Replacing MoveFileEx, with
> > unlink (to);
> > rename (from, to);
> >
> > #if defined (WIN32) &&! defined (__ CYGWIN__)
> > unlink(to);
> > while (rename (from, to)! = 0)
> > #else
> > while (rename (from, to) <0)
> > #endif
> >
> > The log messages have disappeared.
> >
> > I suspect that if the target (to) file exists, MoveFileEx, it is failing
> to
> > rename, even with the flag enabled.
> >
> > Windows have the native rename function (
> >
> https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/rename-wrename?view=vs-2019
> > )
> > However, it fails if the target (to) file exists.
> >
> > Question, is it acceptable delete the target file, if it exists, to
> > guarantee the rename?
>
> I don't think so - the idea behind writing $f.tmp and renaming it to $f is
> that
> it's *atomic*.
>
"atomic" here means, rename operation only, see.
1. create $f.tmp
2. write
3. close
4. rename to $f

>
> I found an earlier thread addressing the same problem - we probably both
> should've found this earlier.
> https://commitfest.postgresql.org/27/2230/
>
> https://www.postgresql.org/message-id/flat/CAPpHfds9trA6ipezK3BsuuOSQwEmESiqj8pkOxACFJpoLpcoNw%40mail.gmail.com#9b04576b717175e9dbf03cc991977d3f

Very interesting.
That link, gave me a light.
https://www.virtualbox.org/ticket/2350
Users report this same situation in production environments with a high
load.
Here I have only one notebook, with 8gb ram, windows 10 64 bits (2004),
msvc 2019 (64 bits).
Without any load, just starting and stopping the server, the messages
appear in the log.

>
>
> That thread goes back to 2017, so I don't think this is a new problem in
> v13.
> I'm not sure why you wouldn't also see the same behavior in v12.
>
Windows Server 2003 (32 bits) with Postgres 9.6 (32 bits), none log (could
rename).`
Windows Server 2016 (64 bits) with Postgres 12 (64 bits), have log (cound
rename).

Yes V12 can confirm, too.

File postgresql-Mon.log:
2019-10-14 11:56:26 GMT [4844]: [1-1] user=,db=,app=,client= LOG: could
not rename temporary statistics file "pg_stat_tmp/global.tmp" to
"pg_stat_tmp/global.stat": Permission denied
File postgresql-Sat.log:
2019-09-28 10:15:52 GMT [4804]: [2-1] user=,db=,app=,client= LOG: could
not rename temporary statistics file "pg_stat_tmp/global.tmp" to
"pg_stat_tmp/global.stat": Permission denied
2019-10-05 12:01:23 GMT [4792]: [1-1] user=,db=,app=,client= LOG: could
not rename temporary statistics file "pg_stat_tmp/global.tmp" to
"pg_stat_tmp/global.stat": Permission denied
2019-10-05 23:55:31 GMT [4792]: [2-1] user=,db=,app=,client= LOG: could
not rename temporary statistics file "pg_stat_tmp/global.tmp" to
"pg_stat_tmp/global.stat": Permission denied
File postgresql-Sun.log:
2019-10-06 07:43:36 GMT [4792]: [3-1] user=,db=,app=,client= LOG: could
not rename temporary statistics file "pg_stat_tmp/global.tmp" to
"pg_stat_tmp/global.stat": Permission denied
File postgresql-Tue.log:
2019-10-01 07:31:46 GMT [4804]: [3-1] user=,db=,app=,client= LOG: could
not rename temporary statistics file "pg_stat_tmp/global.tmp" to
"pg_stat_tmp/global.stat": Permission denied
2019-10-22 14:44:25 GMT [3868]: [1-1] user=,db=,app=,client= LOG: could
not rename temporary statistics file "pg_stat_tmp/global.tmp" to
"pg_stat_tmp/global.stat": Permission denied
File postgresql-Wed.log:
2019-10-02 22:20:52 GMT [4212]: [1-1] user=,db=,app=,client= LOG: could
not rename temporary statistics file "pg_stat_tmp/global.tmp" to
"pg_stat_tmp/global.stat": Permission denied
2019-10-09 11:05:02 GMT [3712]: [1-1] user=,db=,app=,client= LOG: could
not rename temporary statistics file "pg_stat_tmp/global.tmp" to
"pg_stat_tmp/global.stat": Permission denied

>
> There's a couple patches in that thread, and the latest patch was rejected.
>
> Maybe you'd want to test them out and provide feedback.
>
I will try.

> BTW, the first patch does:
>
> ! if (filePresent)
> ! {
> ! if (ReplaceFile(to, from, NULL,
> REPLACEFILE_IGNORE_MERGE_ERRORS, 0, 0))
> ! break;
> ! }
> ! else
> ! {
> ! if (MoveFileEx(from, to,
> MOVEFILE_REPLACE_EXISTING))
> ! break;
> ! }
>
> Since it's racy to first check if the file exists, I would've thought we
> should
> instead do:
>
> ret = ReplaceFile()
> if ret == OK:
> break
> else if ret == FileDoesntExist:
> if MoveFileEx():
> break
>
> Or, should we just try to create a file first, to allow ReplaceFile() to
> always
> work ?
>
This seemingly, it works too.
+ while (rename(from, to) != 0 && !MoveFileEx(from, to,
MOVEFILE_REPLACE_EXISTING | MOVEFILE_COPY_ALLOWED))

Attached a patch.

regards,
Ranier Vilela