Re: extension_control_path

Dimitri,

* Dimitri Fontaine (dimitri(at)2ndQuadrant(dot)fr) wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> > The problem I see, however, is that most extensions, by recommendation,
> > set module_pathname = '$libdir/pgfoo', and so relocating the control
> > files will still end up pointing to a not relocated library file.
>
> It's kind of true. Is the phrasing “typically” followed by an example
> really a recommendation though? I though it was more a detailed
> explanation of the way it works.

It probably sufficient to say that nearly all the ones out there are
this way, so I don't know that it really matters if the userbase
consideres it a recommendation or just documentation- that's what
they're doing.

I admit, I've not gone and looked through PGXN explicitly to check this,
but I'm pretty sure I'm right because every one I've ever seen has done
it this way.

> Also, editing the AS '$libdir/foo' occurences from an SQL script is a
> quite very easy thing to do programmatically.

Sure, but I tend to agree with Peter about it not being a terribly great
solution to the problem.

> I mainly see that as a distribution/distributor problem tho.

This comment doesn't really make any sense to me. My hope is to have
some kind of automated process to build .debs for projects on PGXN but
people are still going to be able to download them directly. If what's
in PGXN matches what the debs have, then people downloading directly
will have to modify everything. Perhaps PGXN could be modified to make
it something different for people downloading directly and then the debs
could switch it to $libdir- except, what would they set it to..? And
we'd be asking every single author to make a change to their module,
which is a pretty terrible thing to do.

> > In order to address this properly, we need a new directory structure
> > that keeps library files and control files together, similar to how
> > Python, Ruby, etc. install things, and then just one path for everything.
>
> It might be true, be it reads to me like you're omiting the directory
> parameter from the control file: the scripts and auxilliary control
> files might be found anywhere else on the file system already.

This is true and Debian puts the control/sql files into a different
directory than the .so files, of course. Still, the issue here is how
we find the .so files- the user *has* to tell us where the control file
is, if it isn't in the default location, and the assumption (default?)
is then that the .sql files are co-located with them. It's at that
point when we get to the point of trying to figure out what $libdir is
and it strikes me that when users use this approach the expectation will
be that it's in the same directory as the control and .sql files.
Making that 'just work' has some definite advantages, though I also
share the concern of others (who I don't think have chimed in on this
particular topic thus far, so I'm really just guessing and it might be
just me) that we probably don't want to just start having the ability to
load .so files from anywhere- except, well, the superuser can already do
that anyway using C functions, so perhaps that isn't really a concern?

I'm still worired about the *conflicts* issue, where you might have a
.so from a -contrib install in $libdir, with .sql/.control files from
downloading the extension itself, hence my suggestion to actually use a
different namespace for extensions which exist outside of the normal
directories. I'm thinking that would also help people doing
dump/restore to realize when they forget to pull in the 'updated' module
from PGXN rather than using the shipped-with-distro version. Of course,
you still run the risk that such a dump might not work upon restore
(same as we have today w/ C functions whose .so's have vanished- try to
call them and you get an error back), though perhaps in this case we'd
end up with a dump that won't restore, right..? That's not ideal
either, of course, but going back to the C functions again, it's
possible you could have a functional index using a custom C functions
and it's the same issue all over again.

> Again, my view is that if you want to do things in a non-standard way
> then you need to tweak the control file and maybe the script files. It's
> a distribution problem, and I'm solving it in an extra software layer.

That's a pretty reasonable option for this specific issue, but it
doesn't address the duplication problem, which worries me.

> PostgreSQL is very flexible about where to organise extension files
> currently, *except* for the control file. This patch is providing the
> same level of flexibility to this part. Of course flexibility can be
> seen as creating a mess, but I don't think it's this patch nor
> PostgreSQL core to solve that mess.

Yeah, but we should be trying to avoid practices and configurations
which encourage mess creation. :)

> > Also, the documentation states that this controls the location of the
> > control file, but it of course controls the location of the script files
> > also. That should be made clearer. (It becomes clearer if we just have
> > one path for everything. ;-) )
>
> Again, we have directory = 'whatever' in the control file to control
> where to find the script files. I'm not sure your "of course" follows.
> Will still edit docs.

Yeah, but it seems to be pretty rarely used and the expectation is that
the .sql files resides in the same directory. I think what we're
looking for here, in some ways, is for that default for .so's to work
out the same- except that right now, the users seem to all default to
sticking in $libdir. Anyhow, I'd like to hear about how the duplication
issue might be addressed (or your thoughts on my proposal) rather than
worry overmuch about a relatively small tweak which we could make later
(but before release) to address the $libdir problem; and, as you say,
you could solve that with a distribution framework for PGXN anyway.

Thanks,

Stephen