Re: pg_basebackup failed to read a file

Mike Cardwell <mike(dot)cardwell(at)hardenize(dot)com> writes:
> pg_basebackup: could not get write-ahead log end position from server:
> ERROR:  could not open file "./postgresql.conf~": Permission denied

> Now, I know what this error means. There was a root owned file at
> "/var/lib/pgsql/10/data/postgresql.conf~" which contained an old
> version of our postgres config and was not readable by the postgres
> user. I'll delete this file and try again. However, in the mean time: I
> feel like it would be useful for pg_basebackup to check that it has
> read access to all of the existing files in the source directory at the
> start, before it begins it's copy.

That seems like a pretty expensive thing to do, if there are lots of
files ... and you'd still end up failing, so it's not moving the ball
very far.

More generally, this seems closely related to bug #14999 [1]
which concerned pg_rewind's behavior in the face of unexpected file
permissions within the data directory. We ended up not doing anything
about that except documenting it, which I wasn't very satisfied with,
but the costs of doing better seemed to exceed the benefits.

It'd be nice to have a more coherent theory about what needs to be copied
or not, and not fail on files that could simply be ignored. Up to now
we've resisted having any centrally defined knowledge of what can be
inside a PG data directory, but maybe that bullet needs to be bitten.

regards, tom lane

[1] https://www.postgresql.org/message-id/flat/20180104200633.17004.16377%40wrigleys.postgresql.org