Re: pg_combinebackup does not detect missing files

On Fri, May 17, 2024 at 6:14 PM David Steele <david(at)pgmasters(dot)net> wrote:
> Then intentionally corrupt a file in the incr backup:
>
> $ truncate -s 0 test/backup/incr1/base/5/3764_fsm
>
> In this case pg_verifybackup will error:
>
> $ pg_verifybackup test/backup/incr1
> pg_verifybackup: error: "base/5/3764_fsm" has size 0 on disk but size
> 24576 in the manifest
>
> But pg_combinebackup does not complain:
>
> $ pg_combinebackup test/backup/full test/backup/incr1 -o test/backup/combine
> $ ls -lah test/backup/combine/base/5/3764_fsm
> -rw------- 1 dev dialout 0 May 17 22:08 test/backup/combine/base/5/3764_fsm
>
> It would be nice if pg_combinebackup would (at least optionally but
> prefferrably by default) complain in this case rather than the user
> needing to separately run pg_verifybackup.

My first reaction here is that it would be better to have people run
pg_verifybackup for this. If we try to do this in pg_combinebackup,
we're either going to be quite limited in the amount of validation we
can do (which might lure users into a false sense of security) or
we're going to make things quite a bit more complicated and expensive.

Perhaps there's something here that is worth doing; I haven't thought
about this deeply and can't really do so at present. I do believe in
reasonable error detection, which I hope goes without saying, but I
also believe strongly in orthogonality: a tool should do one job and
do it as well as possible.

--
Robert Haas
EDB: http://www.enterprisedb.com