Re: base backup vs. concurrent truncation

From: Greg Stark <stark(at)mit(dot)edu>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: base backup vs. concurrent truncation
Date: 2023-05-10 21:02:51
Message-ID: CAM-w4HMDSTsuVbv3LTeCkgzWawiWD7Y6FVDnefhHs7wmGgO3PQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Including the pre-truncation length in the wal record is the obviously
solid approach and I none of the below is a good substitution for it.
But....

On Tue, 25 Apr 2023 at 13:30, Andres Freund <andres(at)anarazel(dot)de> wrote:
>
> It isn't - but the alternatives aren't great either. It's not that easy to hit
> this scenario, so I think something along these lines is more palatable than
> adding a pass through the entire data directory.

Doing one pass through the entire data directory on startup before
deciding the directory is consistent doesn't sound like a crazy idea.
It's pretty easy to imagine bugs in backup software that leave out
files in the middle of tables -- some of us don't even have to
imagine...

Similarly checking for a stray next segment whenever extending a file
to maximum segment size seems like a reasonable thing to check for
too.

These kinds of checks are the kind of paranoia that catches filesystem
bugs, backup software bugs, cron jobs, etc that we don't even know to
watch for.

--
greg

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message David Rowley 2023-05-10 22:27:28 Re: benchmark results comparing versions 15.2 and 16
Previous Message Jehan-Guillaume de Rorthais 2023-05-10 21:00:31 Re: Unlinking Parallel Hash Join inner batch files sooner