Re: Speed up the removal of WAL files

On Sat, Nov 18, 2017 at 2:57 AM, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
> On Fri, Nov 17, 2017 at 5:20 PM, Tsunakawa, Takayuki
> <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com> wrote:
>> From: Kyotaro HORIGUCHI [mailto:horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp]
>>> The orinal code recycles some of the to-be-removed files, but the patch
>>> removes all the victims. This may impact on performance.
>>
>> Yes, I noticed it after submitting the patch and was wondering what to do. Thinking simply, I think it would be just enough to replace durable_unlink/durable_rename in RemoveXLogFile() with unlink/rename, and sync the pg_wal directory once in RemoveNonParentXlogFiles() and RemoveOldXlogFiles(). This will benefit the failover time when fast promotion is not performed. What do you think?

Note that durable_rename() also flushes the origin file to make sure
that it does not show up again after a crash.

> It seems not good idea to just replace durable_rename() with rename()
> in RemoveOldXlogFiles()->RemoveXlogFiles()->InstallXLogFileSegment().
> Because that change seems to be able to cause the following problem.

If archiving is enabled, RemoveOldXlogFiles() would create a .ready
flag for all segments that have reappeared. Oops, it archived again.
--
Michael