Re: jsonb and nested hstore

On 02/28/2014 06:27 AM, Robert Haas wrote:
> Taken individually, none of those decisions seem crazy, but taken
> together it's pretty weird. Instead of inventing a new type (jsonb)
> designed from the ground up to do what we want, we're, well, we're
> doing what Christophe says: creating our own proprietary hierarchical
> type and then making the hierarchical type everyone else uses depend
> on it. Described in those terms, it's hard for me to believe that
> anyone here thinks that's not a strange thing to do.

It certainly seems like a strange thing to do to *me*. However, Oleg
and Teodor were doing the heavy lifting on the heirarchical type -- we
wouldn't even be talking about jsonb without it -- and they were very
negative to JSON. As with many things, this reminds me of a story.

When the BART subway system was being built for the Bay Area in 1970,
the tunnel was planned to go straight under a particular hardware store
in Berkeley. The hardware store owner was convinced that the
construction would destroy his building and his business, and hit the
state with lawsuit after lawsuit to stop the construction. Eventually,
CALtrans caved and added a curve in that section of the BART tunnel to
go around the location of the hardware store. Forty years later, the
hardware store owner is dead, the hardware store is gone (was gone, in
fact, by 1978), but the curve is still there. And that curve forces
BART trains to slow down by 25mph in a spot which is fairly central to
the whole BART system, thus reducing the overall max capacity of the
entire subway system by 10-15%, and thus making thousands of people a
day late for work for the past 40 years.

I think Robert and Christophe are right: we're building a Berkeley BART
Curve. I think there's two courses of action from here which make sense:

A) We move *all* of the important HStore libraries and operators into
core, and make the hstore extension of them just a mapping of what are
essentially jsonb operators to the hstore type (Christophe's suggestion).

B) We make hstore/jsonb a single extension with two types and all of the
requisite operators etc. (Peter's suggestion).

Reasons for (A):

* In-core jsonb would have strongly enhanced adoption value
* jsonb is liable to become one of our most-used types and it would be
strange for it not to be in core
* binary JSON would "just work" for web developers
* the only reason nested hstore is an extension is because hstore was an
extension and we need an upgrade path
* This is essentially the decision we collectively made in November, for
fairly well-argued reasons, and what Andrew has spent 3 months implementing.

Reasons against (A):

* Having a core type and an extension share code is strange.
* Implicit casts between a core type and an extension could cause issues.

Reasons for (B):

* Conceptually simpler.
* Makes a certain degree of bugginess/unfinishedness more acceptable.

Reasons against (B):

* Users would get tripped up by "first, install the postgresql-contrib
package, then do CREATE EXTENSION hstore"
* As cited, many sysadmins block the install of the -contrib package.
* Performance issues for psycopg2 and other drivers which need to look
up type information on each call.
* This requires larger changes to the existing patch, which likely means
missing the bus for 9.4 (and you've seen my blog about that)

Here's the point in particular which makes me very hesitant to endorse
(B) as a solution:

* Once created as an extension, there is no path to ever making jsonb a
core type.

... as long as we have no way to ever move types between extensions and
core, any decision we make on where a type belongs is permanent. This
is one of the things which Andrew's proposal of a Type Registry last
year was intended to solve, but -hackers soundly rejected that proposal,
so we're currently stuck in the proverbial polluted estuary.

My cause, as everyone knows, is adoption. Given that, I'm pretty
strongly in favor of proposal (A); I think a jsonb type which "just
works" will drive twice the adoption that one you have to remember to
install does.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com