Re: PostgreSQL extensions packaging

Hi,

Le jeudi 24 juillet 2008, Tom Dunstan a écrit :
> I guess that means you missed both the original discussion at
> http://archives.postgresql.org/pgsql-hackers/2008-04/msg00132.php and
> my initial patch in that direction and subsequent discussion at
> http://archives.postgresql.org/pgsql-hackers/2008-04/msg00132.php then

Thanks for the links, I've read a little down there now :)

> There were two core components to my idea of modules/packages:

First reading makes me think your proposal is all about having a user-visible
management of modules, which in my proposal are a part of packages, and a
much needed one.
So it seems to me both proposals are complementary, in that I didn't go in any
detail about how to manage this module part of a package declaration, and it
looks like your work is all about this.

Where we're trying to solve the same issue(s) is on the OS level packaging.

> - Separation of installation at an OS level (RPM/yum, deb/dpkg, MSI
> installer etc) and installation into a database. The intention was a)
> to standardize package installation generally so that users didn't
> have to read n different sets of installation instructions for n
> different packages, and b) so that a db owner could install into their
> own database any module that had been installed on the system, even if
> that might include e.g. C functions that they otherwise would not be
> able to install without being a superuser.

I'm proposing that PostgreSQL includes a source level package management
toolset, and the OS distributions take advantage of it to release binary
packages easy to install, as it's done now with make && make install (using
PGXS) at PG level.

As you're saying, OS install means the same thing as PGXS make install, that
is having the .so and .sql files at the right place and in the right format.
So even if PostgreSQL was to propose a source level integration with pg_pkg
add <package>, distributions would still be left with the binary packaging
work.

As for the database level installation, I think this is best done by
PostgreSQL itself this time, I'd much prefer the distributions not to bother
with pg_pkg install <package> <database>.
Of course, debian wrapper scripts would certainly repackage this in order for
the user to choose which cluster to target here.

> - Have dependency tracking so that pg_dump could emit e.g. "LOAD
> MODULE foo;" rather than all the different instructions to recreate
> the module.

That could be what the module section of create package means internally. I
don't foresee a need for separating module only management stuff out of
package, but I'm all ears :)

> So the proposed installation procedure would be more along the lines of:
>
> yum install postgresql-module-postgis
> echo "load module postgis" | psql mydb

Agreed, but with those little differences:
- PostgreSQL provides pg_pkg add to distribution to ease binary packaging
- apt-get install postgresql-module-8.3-prefix
- and either
$ pg_pkg install prefix mydb
or
$ psql -c "INSTALL PACKAGE prefix" mydb

> My intention was to use whatever native package manager was
> appropriate for your distro rather than trying to recreate CPAN,
> although some people in the original discussion wanted to go down that
> route.

I know nothing about CPAN, but I hope offering tools for packagers to ease
their work is a good idea. Plus this allows for the PostgreSQL project
approved extensions, -core level quality, reviewed code at an easy to grasp
place.
And it allows advanced user, who compile their PostgreSQL theirself, to still
benefit from a level of OS integration for packages.

> The patch that I provided didn't do any of the dependency stuff yet -
> I had been investigating various ways to do it automagically, although
> I haven't worked on it for a little while. It may be that the straight
> forward explicit declaration that you have here is a better way to do
> it.

It seems to me that your patch would certainly be a step towards implementing
my idea of a package.

> I didn't have versioning and interdependencies between modules yet,
> although it's an obvious extension to the idea.

And a much necessary one. As soon as we have a SQL level object for modules,
with oids in the catalog and all, we surely are able to add entries in
pg_depend about this?

> > A package can also host variables, which visibility are
> > package global: any SQL into the package can refer directly to package
> > variables.
>
> That was way out of scope for my more modest suggestion - I certainly
> wasn't going to change pl/pgsql semantics. For example, how do those
> variables behave upon a transaction rollback?

No idea yet, I just saw that Oracle packages host package level global
variables, and I guess it would work the same as a SET [LOCAL] GUC, except
you could only see the variable from objects within the package.

> This turns into recreating CPAN. I like the idea of a "blessed" set of
> packages, but would rather not require all postgresql users to have a
> full build environment (especially on windows) and have to replace
> their native packaging solution.

As said before, I'm thinking about providing this pg_pkg add as a packager
facility, not replacing binary distributions, but still available for
advanced user.

> I think that we can come up with a package/module format that allows
> installation at the OS level without demanding a whole set of download
> / build machinery.

I think this part is up to distributions.

Oh, and... well, I don't think I'm in a position to write the code to make
this packaging idea a reality. I'm willing to contribute as a non-hacker here
by helping to define a user-level specification which would need to find a
developer.

Hope this helps, regards,
--
dim