Re: RFC: Extension Packaging & Lookup

On Oct 29, 2024, at 13:16, Paul Ramsey <pramsey(at)cleverelephant(dot)ca> wrote:

> An apposite choice, since it not only demonstrates depending on a common system library, it also demonstrates the way these things loop on each other, as curl then depends on libssl, which postgres also depends on.

Ooh, yeah, vicious!

> Relative rpaths as I have seen them are relative to the executable or library in which they are defined (as far as I know, I’m not a dylib expert by any stretch). The implication is that extension.so could have an rpath=. and dependent dylibs sitting next to it. This is how, for example, cmake out-of-tree builds can run tests against the newly built library before it’s installed,.. the test execs have an rpath of ../lib on them.

Oh that’s super interesting. Will be worth trying. I was hoping to avoid having to set rpath in every extension, though, but maybe that’d be the way forward.

The issue, though, is a tree of dependencies. Would you really want to include both libcurl and OpenSSL in a pgsql-http binary distribution package --- especially since Postgres itself will be using its own OpenSSL package? To Christophe’s point, I think we might want to delegate the provisioning of dependency DSOs to the system package manager.

>> Yeah, I think the issue will be to figure out how to manage OS package-provided system dependencies in immutable environments like a Docker container. I suspect some combination of -rpath compiled into Postgres and mounting individual DSO files not included in the base image will be the way to go.
>
> Do you see immutable images as the main deployment path going forward? I’m not container-pilled yet, so it still feels like a niche concern. Meanwhile being able to “add an extension that my cloud provider doesn’t support yet” feels quite vital.

I think it will become increasingly common. At first I thought it was just Docker/Kubernetes --- and in fairness, a lot of orgs are running Postgres in such environments these days. Some Postgres as a Service companies exclusively use Kubernetes (my employer, Tembo, is one).

But then Tobias Bussmann pointed out[1] that Postgres.app for macOS has the same issue: it has to be immutable in order to validate the app via Apple’s provisioning certificate. If you change the contents of the Postgres.app bundle, you can no longer run it! It would not surprise me if there were other examples, and that the pattern becomes increasingly common.

Best,

David

[1]: https://justatheory.com/2024/03/mini-summit-two/