Re: RFC: Additional Directory for Extensions

Hi David, thanks for testing!

On Wed, Mar 19, 2025 at 2:29 PM David E. Wheeler <david(at)justatheory(dot)com> wrote:
>
> First thing I notice is that prefix= uses the magic to insert
> “postgresql” into the path if it’s not already there:
>
> ``` console
> ❯ make PG_CONFIG=~/dev/c/postgres/pgsql-devel/bin/pg_config prefix=/Users/david/Downloads install
> /opt/homebrew/bin/gmkdir -p '/Users/david/Downloads/share/postgresql/extension'
> /opt/homebrew/bin/gmkdir -p '/Users/david/Downloads/share/postgresql/extension'
> /opt/homebrew/bin/gmkdir -p '/Users/david/Downloads/share/doc//postgresql/extension'
> /opt/homebrew/bin/ginstall -c -m 644 .//pair.control '/Users/david/Downloads/share/postgresql/extension/'
> /opt/homebrew/bin/ginstall -c -m 644 .//sql/pair--0.1.2.sql .//sql/pair--unpackaged--0.1.2.sql '/Users/david/Downloads/share/postgresql/extension/'
> /opt/homebrew/bin/ginstall -c -m 644 .//doc/pair.md '/Users/david/Downloads/share/doc//postgresql/extension/‘
> ```
>
> I think this should at least be documented, but generally feels
> unexpected to me. I’ve attached a patch that fleshes out the docs,
> along with an example of setting `extension_control_path` and
> `dynamic_library_path` to use the locations. It might not have the
> information right about the need for “postgresql” or “pgsql” in the
> path.

Did you miss to attach the patch?

> Everything else works very nicely except for extensions that use the
> Makefile `MODULEDIR` variable to install all of the share files except
> the control file into a particular directory, and the `directory` in
> the control file so that the files can be found. Here’s semver[2],
> which has both:
>
> ```console
> ❯ make PG_CONFIG=~/dev/c/postgres/pgsql-devel/bin/pg_config prefix=/Users/david/Downloads/postgresql install
> /opt/homebrew/bin/gmkdir -p '/Users/david/Downloads/postgresql/share/extension'
> /opt/homebrew/bin/gmkdir -p '/Users/david/Downloads/postgresql/share/semver'
> /opt/homebrew/bin/gmkdir -p '/Users/david/Downloads/postgresql/lib'
> /opt/homebrew/bin/gmkdir -p '/Users/david/Downloads/postgresql/share/doc//semver'
> /opt/homebrew/bin/ginstall -c -m 644 .//semver.control '/Users/david/Downloads/postgresql/share/extension/'
> /opt/homebrew/bin/ginstall -c -m 644 .//sql/semver--0.10.0--0.11.0.sql .//sql/semver--0.11.0--0.12.0.sql .//sql/semver--0.12.0--0.13.0.sql .//sql/semver--0.13.0--0.15.0.sql .//sql/semver--0.15.0--0.16.0.sql .//sql/semver--0.16.0--0.17.0.sql .//sql/semver--0.17.0--0.20.0.sql .//sql/semver--0.2.1--0.2.4.sql .//sql/semver--0.2.4--0.3.0.sql .//sql/semver--0.20.0--0.21.0.sql .//sql/semver--0.21.0--0.22.0.sql .//sql/semver--0.22.0--0.30.0.sql .//sql/semver--0.3.0--0.4.0.sql .//sql/semver--0.30.0--0.31.0.sql .//sql/semver--0.31.0--0.31.1.sql .//sql/semver--0.31.1--0.31.2.sql .//sql/semver--0.31.2--0.32.0.sql .//sql/semver--0.32.0--0.32.1.sql .//sql/semver--0.32.1--0.40.0.sql .//sql/semver--0.32.1.sql .//sql/semver--0.40.0.sql .//sql/semver--0.5.0--0.10.0.sql .//sql/semver--unpackaged--0.2.1.sql .//sql/semver.sql '/Users/david/Downloads/postgresql/share/semver/'
> /opt/homebrew/bin/ginstall -c -m 755 src/semver.dylib '/Users/david/Downloads/postgresql/lib/'
> /opt/homebrew/bin/gmkdir -p '/Users/david/Downloads/postgresql/lib/bitcode/src/semver'
> /opt/homebrew/bin/gmkdir -p '/Users/david/Downloads/postgresql/lib/bitcode'/src/semver/src/
> /opt/homebrew/bin/ginstall -c -m 644 src/semver.bc '/Users/david/Downloads/postgresql/lib/bitcode'/src/semver/src/
> cd '/Users/david/Downloads/postgresql/lib/bitcode' && /opt/homebrew/Cellar/llvm/19.1.7_1/bin/llvm-lto -thinlto -thinlto-action=thinlink -o src/semver.index.bc src/semver/src/semver.bc
> /opt/homebrew/bin/ginstall -c -m 644 .//doc/semver.mmd '/Users/david/Downloads/postgresql/share/doc//semver/‘
> ```
>
> Following `MODULEDIR=semver`, it puts the SQL files into
> `share/semver/` instead of `share/extension/`, as expected, but then,
> even though the control file has `directory=semver`, it can’t load
> them:
>
> ```pgsql
> david=# create extension semver;
> ERROR: could not open directory "/Users/david/dev/c/postgres/pgsql-devel/share/semver": No such file or directory
> ```

I've managed to reproduce the issue. The problem is on get_ext_ver_list
which calls get_extension_script_directory that try to search for .sql
files only on $sharedir.

> Looks like it’s only looking in the `semver` subdirectory under
> $libdir and not the whole path.
>
> But given that the `directory` variable in the control file can be a
> full path, I don’t see that there’s much of a way to generalize a
> solution. I guess there are three options:
>
> 1. If directory is a full path, try to load the files there. It
> probably already works that way, though I haven’t tired it.
>
Yes, if the directory is a full path it try to load the files from
there. It is implemented on get_extension_script_directory.

> 2. If the directory is not a full path, check for it under each
> directory in `extension_control_path`? But no, that points to
> `share/extension`, not `share`, so it can’t really searched unless it
> also lops off `extension` from the end of each path.

Maybe we could make the "extension" part of the extension control path
explicitly, like Peter has mentioned in his first patch version [1]?.
If "directory" is not set we could use "extension" otherwise use the
"directory" as a path suffix when searching on extension_control_path?

[1] https://www.postgresql.org/message-id/0d384836-7e6e-4932-af3b-8dad1f6fee43%40eisentraut.org

--
Matheus Alcantara