Re: archive status ".ready" files may be created too early

From: Andres Freund <andres(at)anarazel(dot)de>
To: "Bossart, Nathan" <bossartn(at)amazon(dot)com>
Cc: "alvherre(at)alvh(dot)no-ip(dot)org" <alvherre(at)alvh(dot)no-ip(dot)org>, Robert Haas <robertmhaas(at)gmail(dot)com>, Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, "x4mmm(at)yandex-team(dot)ru" <x4mmm(at)yandex-team(dot)ru>, "a(dot)lubennikova(at)postgrespro(dot)ru" <a(dot)lubennikova(at)postgrespro(dot)ru>, "hlinnaka(at)iki(dot)fi" <hlinnaka(at)iki(dot)fi>, "matsumura(dot)ryo(at)fujitsu(dot)com" <matsumura(dot)ryo(at)fujitsu(dot)com>, "masao(dot)fujii(at)gmail(dot)com" <masao(dot)fujii(at)gmail(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: archive status ".ready" files may be created too early
Date: 2021-09-01 00:23:47
Message-ID: 20210901002347.2u6tuqh53ulkl3wo@alap3.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 2021-08-31 23:31:15 +0000, Bossart, Nathan wrote:
> On 8/31/21, 1:30 PM, "Andres Freund" <andres(at)anarazel(dot)de> wrote:
> > On 2021-08-31 18:09:36 +0000, Bossart, Nathan wrote:
> >> What appears to happen in this case is that bgwriter eventually creates a
> >> xl_running_xacts record and nudges walwriter to flush it to disk, at which
> >> point the .ready file(s) will be created. That's admittedly a bit fragile.
> >
> > That's not guaranteed to happen. If e.g. the partial record is a checkpoint or
> > a xl_running_xacts, we'll not trigger further WAL writes in the background,
> > unless autovacuum ends up doing something.
>
> Right. Per the attached patch, a simple way to handle that could be
> to teach XLogBackgroundFlush() to flush to the "earliest" segment
> boundary if it doesn't find anything else to do. I think you could
> still miss creating a .ready file for the previous segment in single-
> user mode, though.

Maybe, but this is getting uglier and uglier.

I think patch should be reverted. It's not in a state that's appropriate for
the backbranches.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2021-09-01 00:24:50 Re: pg_receivewal starting position
Previous Message Bossart, Nathan 2021-08-31 23:31:15 Re: archive status ".ready" files may be created too early