Re: make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.

On 2021/03/27 2:14, Andres Freund wrote:
> Hi,
>
> On 2021-03-26 09:27:19 +0900, Masahiro Ikeda wrote:
>> On 2021/03/25 19:48, Fujii Masao wrote:
>>> Yes. So we should wait for the shared memory stats patch to be
>>> committed before working on walreceiver stats patch more?
>>
>> Yes, I agree.
>
> Agreed.

OK. And I withdraw this thread since the shared memory stats patch will solve
this problem too.

> One thing that I didn't quite see discussed enough in the thread so far:
> Have you considered what regressions the stats file removal in immediate
> shutdowns could cause? Right now we will - kind of accidentally - keep the
> stats file for immediate shutdowns, which means that autovacuum etc will
> still have stats to work with after the next start. After this, not so
> much?

Yes([1]). Although we keep the stats file for immediate shutdowns, IIUC, we'll
remove them in redo phase via pgstat_reset_all() anyway. So, this means that
autovaccum etc can't use the stats after the next start.

FWIW, the issue is discussed([2]). The consensus is that we need to change the
behavior removing all stats files even if server crash is occurred because
autovacuum uses the stats. There are some ideas, for example writing the stats
files every X minutes (using wal or another mechanism). Then, the startup
process can restore the stats with slightly low freshness even if a crash
occurs. But, it needs to find a way how to handle the stats files when
deleting on PITR rewind.

This is not solved yet. If my understanding is correct, the shared memory
stats patch didn't consider to handle the issue yet, but to solve it makes the
patch more complicated... I think it's better to work on the issue after the
base patch of the shared memory stats is committed.

[1]
https://www.postgresql.org/message-id/c96d8989100e4bce4fa586064aa7e0e9%40oss.nttdata.com
[2]
https://www.postgresql.org/message-id/flat/0A3221C70F24FB45833433255569204D1F5EF25A%40G01JPEXMBYT05

Regards,
--
Masahiro Ikeda
NTT DATA CORPORATION