Re: Review: extension template

Salut Dimitri,

On 07/09/2013 12:40 PM, Dimitri Fontaine wrote:
> Markus Wanner <markus(at)bluegap(dot)ch> writes:
>> Or how do you think would pg_restore fail, if you followed the mental
>> model of the template?
>
> # create template for extension foo version 'x' as '';
> # create extension foo;
> # alter template for extension foo rename to bar;
>
> $ pg_dump | psql
>
> And now it's impossible to CREATE EXTENSION foo, because there's no
> source to install it from available. I think we should actively prevent
> that scenario to happen in the field (current patch prevents it).

I see. You value that property a lot.

However, I don't think the plain ability to create an extension is quite
enough to ensure a consistent restore, though. You'd also have to ensure
you recreate the very *same* contents of the extension that you dumped.

Your patch doesn't do that. It seems to stop enforcing consistency at
some arbitrary point in between. For example, you can ALTER a function
that's part of the extension. Or ALTER TEMPLATE FOR EXTENSION while an
extension of that version is installed.

> Usually what you do when you want to change an extension is that you
> provide an upgrade script then run it with ALTER EXTENSION UPDATE.

As a developer, I often happen to work on one and the same version, but
test multiple modifications. This ability to change an extension
"on-the-fly" seems pretty valuable to me.

> The current file system based extensions allow you to maintain
> separately the files foo--1.0.sql and foo--1.2.sql, and you don't need
> to copy a current version of the whole extension away before hacking the
> new version.
>
> The Extension Template facility in the current patch allows you to do
> much the same

Sure. I'm aware of that ability and appreciate it.

> I see that you've spent extra time and effort to better understand any
> community consensus that might exist around this patch series, and I
> want to say thank you for that!

Thank you for your patience in pursuing this and improving user
experience with extensions. I really appreciate this work.

> Basically what I'm saying in this too long email is that I need other
> contributors to voice-in.

I fully agree. I don't want to hold this patch back any further, if it's
just me.

>> I really recommend to rename the feature (and the
>> commands), in that case, though. We may rather call the existing
>> file-system thingie an "extension template", instead, as it becomes a
>> good differentiator to what you're proposing.
>
> Any proposals?

"Inline extension" is a bit contradictory. Maybe "managed extension"
describes best what you're aiming for.

In a similar vein, "out-of-line extension" sounds redundant to me, so
I'd rather characterize the file-system thingie as "extension templates"
wherever a clear distinction between the two is needed.

>> How about ALTER EXTENSION ADD (or DROP)? With the link on the
>> "template", you'd have to prohibit that ALTER as well, based on the
>> exact same grounds as the RENAME: The installed extension would
>> otherwise differ from the "template" it is linked to.
>
> You're "supposed" to be using that from within the template scripts
> themselves. The main use case is the upgrade scripts from "unpackaged".

Agreed. The documentation says it's "mainly useful in extension update
scripts".

> I could see foreclosing that danger by enforcing that creating_extension
> is true in those commands.

For managed extensions only, right? It's not been prohibited for
extensions so far.

>> See how this creates an animal pretty different from the current
>> extensions? And IMO something that's needlessly restricted in many ways.
>
> Well really I'm not convinced. If you use ALTER EXTENSION ADD against an
> extension that you did install from the file system, then you don't know
> what will happen after a dump and restore cycle, because you didn't
> alter the files to match what you did, presumably.

Yeah. The user learned to work according to the template model. Maybe
that was not the best model to start with. And I certainly understand
your desire to ensure a consistent dump & restore cycle. However...

> If you do the same thing against an extension that you did install from
> a catalog template, you just managed to open yourself to the same
> hazards…

... I think the current patch basically just moves the potential
hazards. Maybe these moved hazards are less dramatic and justify the
move. Let's recap:

In either case (template model & link model) the patch:

a) guarantees to restore the template scripts and settings of
all managed extensions

With the link model (and v9 of your patch, which includes the RENAME
fix, and pretending there are no other bugs):

b) it guarantees *some* revision of an extension version (identified
by name and version) that has been created at dump time can be
restored from a template from the dump

If you'd additionally restrict ALTER EXTENSION ADD/DROP as well as ALTER
TEMPLATE FOR EXTENSION AS ..., you'd also get:

c) it guarantees the set of objects created by managed extensions
is restored to exactly the same set as the one that got dumped

So far, neither template nor link model guarantee:

d) the contents (functions, types, operators, ...) of the actual
extension installed matches the contents before the dump

In the template model, you'd get the following benefit:

e) templates (esp. upgrade scripts) can be used in a mix and match
fashion, it doesn't matter whether in-line or out-of-line was
used, you can always apply update scripts from any of the two.

Given d), I personally don't value b) and c) all that much and rather
accuse them of pretending a bit of a false safety.

I'm pretty keen about a) and really appreciate that part of Dimitri's
patch. I see great value in e) as well.

What do others think?

Regards

Markus Wanner