Re: Remove_temp_files_after_crash and significant recovery/startup time

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: "McCoy, Shawn" <shamccoy(at)amazon(dot)com>
Cc: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Remove_temp_files_after_crash and significant recovery/startup time
Date: 2021-09-10 21:32:58
Message-ID: 4098560.1631309578@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

"McCoy, Shawn" <shamccoy(at)amazon(dot)com> writes:
> I noticed that the new parameter remove_temp_files_after_crash is currently set to a default value of "true" in the version 14 release. It seems this was discussed in this thread [1], and it doesn't look to me like there's been a lot of stress testing of this feature.

Probably not ...

> In our fleet there have been cases where we have seen hundreds of thousands of temp files generated. I found a case where we helped a customer that had a little over 2.2 million temp files. Single threaded cleanup of these takes a significant amount of time and delays recovery. In RDS, we mitigated this by moving the pgsql_tmp directory aside, start the engine and then separately remove the old temp files.

TBH, I think the thing to be asking questions about is how come you had so
many temp files in the first place. Sounds like something is misadjusted
somewhere.

regards, tom lane

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Tomas Vondra 2021-09-10 21:57:24 Re: Remove_temp_files_after_crash and significant recovery/startup time
Previous Message David Zhang 2021-09-10 21:23:48 Re: ORDER BY pushdowns seem broken in postgres_fdw