Re: RFC: Extension Packaging & Lookup

On Oct 29, 2024, at 12:23, Paul Ramsey <pramsey(at)cleverelephant(dot)ca> wrote:

> Thanks for this, David,

🤘🏻

> This of course is the area that worries the heck out of me, as someone with extensions that includes not just single system dependencies but long chains of them (depending on GDAL draws in a huge tree).

Yeah. I cited pgsql-http as a simple place to start, on the assumption that once we figure out how to properly configure things for one DSO, it the pattern should work for any of them in a tree.

> I’m unsure if it will work, but I have wondered if building out the dependencies to install right next to the DSO, and giving the DSO an rpath of “.” would achieve the effect we are looking for.

Given the security issues with library paths, I’d guess that relative paths are verboten. But also, Postgres does not `cd` into an extension directory before loading it, AFAIK.

> It’s unfortunate (DY)LD_LIBRARY_PATH is dead and dying, but there we are. The trouble I see with somehow coercing the system to load a local copy of system libraries is for (a) common system libs that PostgreSQL itself might be linking (libssl, for example) that then will end up with symbol collisions between the copy loaded by postgres and the copy loaded by the DSO and (b) same thing but for different extensions with the same dependencies.

Yeah, this is why people tend to depend on system dependencies loaded from well-known paths, so libssl will always load the same DSO. I imagine the use of LD_LIBRARY_PATH can cause issues today.

> I guess I cannot shake the idea that a lot of interesting extensions are going to have interesting system dependencies, that “exposing an interesting library to postgres” has a high value for an integration system like PostgreSQL.

Yeah, I think the issue will be to figure out how to manage OS package-provided system dependencies in immutable environments like a Docker container. I suspect some combination of -rpath compiled into Postgres and mounting individual DSO files not included in the base image will be the way to go.

Best,

David