Re: [PATCH] Add transforms feature

Hi,

Allow me to temporarily skip important questions that you asked so that
we can focus on the main problem here. As soon as we decide how to
handle any kind of selectivity for the transforms, then I'm back to
answering the other things.

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> Let's review that, as there as been some confusion about that. The SQL
> standard syntax is

Thanks for that. I see that you reused parts of the concept and keywords
and implemented something specific to PostgreSQL from there. As there's
no collision within the command sets, I think that problem is cleared.

> I had proposed disallowing installing a transform that would affect
> existing functions. That was rejected or deemed unnecessary. You can't
> have it both ways. ;-)

Well I'm not so sure, that's the point.

>> A mechanism allowing for the transform to only be used in some functions
>> but not others might be useful. The simplest such mechanism I can think
>> of is modeled against the PL/Java classpath facility as specified in the
>> SQL standard: you attach a classpath per schema.
>
> Anything that's a problem per-database would also be a problem per
> schema.

But a smaller problem, and as problem get smaller they get easier to
reason about and fix too. In the example given by Robert Haas in the
same thread, you could install extension A in schema A using the
transforms for hstore and plperl and extension B in schema B not using
the same transforms.

I'm not saying using the schema that way is awesome, just that we have
solid precedent in the standard and in pljava, and it looks easy enough
to implement (we already have search_path invalidations IIRC).

> This is a transition problem. Nobody is required to install the
> transforms into their existing databases. They probably shouldn't.

I disagree.

> How many people actually use hstore with PL/Perl or PL/Python now?
> Probably not many, because it's weird.
>
> I like to think about how this works for new development: Here is my
> extension type, here is how it interfaces with languages. Once you have
> established that, you don't want to have to repeat that every time you
> write a function. That's error prone and cumbersome. And anything
> that's set per schema or higher is a dependency tracking and caching mess.

The problem is installing a set of extensions where some of them are
already using the new transform feature and some of them are not. We
need a way to cater with that, I think.

> Also, extension types should work the same as built-in types.
> Eventually, I'd like to rip out the hard-coded data type support in
> PL/Python and replace it with built-in transforms. Even if we don't
> actually do it, conceptually it should be possible. Now if we require

I like that idea a lot. I don't see how you can make it work by default,
as like Robert I think the transition phase you're talking about will
never end.

> "USING TRANSFORM FOR int, bytea" every time, we'd have taken a big step
> back. Effectively, we already have built-in transforms in PL/Python.

For core types only.

> We have added a few more over the years. It's been a bit of a pain from
> time to time. At least, with this feature we'd be moving this decision
> into user space and give people a way to fix things. (Incidentally, if
> you add a lot of transforms, you are probably dealing with a strongly
> typed language. And a strongly typed language is more likely to cleanly
> catch type errors resulting from changes in the transforms.)

I think use space will want to be able to use code written using
different sets of transforms for the same set of types, rather than
being forced into upgrading their whole code at once.

It reminds me of the python 2 to python 3 upgrade path. This is not
solved yet, with libs that are python2 only and others that are python3
only, and OS that would ship some but not others.

We already have the problem that shipping contrib by default is frowned
upon at some places, then they miss out of plenty of awesome extensions.
Transforms with no granularity is only going to make it worse, I think.

So we have to choose the granularity:

- per database (current patch),
- per schema (standard has precedents with pljava,
- per extension (forcing users into using extensions, not good),
- per function (hard to maintain),
- something else.

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support