Re: pgsql: Add TAP tests for pg_verify_checksums

Greetings,

On Sat, Oct 20, 2018 at 01:16 Andres Freund <andres(at)anarazel(dot)de> wrote:

> Hi,
>
> On 2018-10-20 00:25:19 -0400, Stephen Frost wrote:
> > If we are going to decide that we only deal with files in our directories
> > matching these whitelisted patterns, then shouldn’t we apply similar
> logic
> > in things like DROP DATABASE and any other cases where we perform actions
> > in a recursive manner across our database and table space directories?
>
> I'm honestly not sure if you're just trolling at this point. Why would
> anybody reasonable be concerned about DROP DATABASE dropping the
> directory for the database? You're not honestly suggesting that anybody
> would write an extension or anything like that that stores data in the
> wrong database's directory, right? Other iterations like fsyncing
> files, copying the entire template database directory, etc are similarly
> harmless or positive.

No, I’m not trolling, what I was trying to do is make the point that this
is moving us away from having a very clear idea of what’s PG’s
responsibility and what we feel comfortable operating on and this new
half-and-half stance where we’ll happily nuke files in a directory that we
didn’t create, and back them up even if we have no idea if they’ll be
consistent at all or not when restored, but we won’t try to check the
checksums on them or do some other set of operations on them.

I suspect it’s pretty clear already, but just to make it plain, I really
don’t like the half-and-half approach and it seems we’re being backed into
this because it happens to fit some specific cases and not because there
was any real thought or design put into supporting these use cases or being
able to extend PG in this way. I do also see real risks with a
whitelisting kind of approach that we end up missing things, and I get that
you don’t see that risk but just stating that doesn’t change my opinion on
it. Based on what you said up thread this whole thing also seems like it
may be just going away anyway- because there’s real design being done to do
this properly and allow PG to be extended in a way that we will know about
what files are associated with what extensions or storage mechanisms and
that seems like just another reason why we shouldn’t be moving to
explicitly support random files being dropped into PG directories.

Thanks,

Stephen