Re: O(n) tasks cause lengthy startups and checkpoints

On Sun, Apr 02, 2023 at 11:42:26AM -0700, Andres Freund wrote:
> Just want to note that I've repeatedly objected to 0002 and 0003, i.e. moving
> serialized logical decoding snapshots and mapping files, to custodian, and
> still do. Without further work it increases wraparound risks (the filenames
> contain xids), and afaict nothing has been done to ameliorate that.

From your feedback earlier [0], I was under the (perhaps false) impression
that adding a note about this existing issue in the commit message was
sufficient, at least initially. I did add such a note in 0003, but it's
missing from 0002 for some reason. I suspect I left it out because the
serialized snapshot file names do not contain XIDs. You cleared that up
earlier [1], so this is my bad.

It's been a little while since I dug into this, but I do see your point
that the wraparound risk could be higher in some cases. For example, if
you have a billion temp files to clean up, the custodian could be stuck on
that task for a long time. I will give this some further thought. I'm all
ears if anyone has ideas about how to reduce this risk.

[0] https://postgr.es/m/20220702225456.zit5kjdtdfqmjujt%40alap3.anarazel.de
[1] https://postgr.es/m/20220217065938.x2esfdppzypegn5j%40alap3.anarazel.de

--
Nathan Bossart
Amazon Web Services: https://aws.amazon.com