Re: jsonb and nested hstore

On Wed, Feb 26, 2014 at 10:42 PM, Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:
>>> Why can't this whole thing be shipped as an extension? It might well
>>> be more convenient to have the whole thing packaged as an extension
>>> than to have parts of it in core and parts of it not in core.
>>
>> That's a good question. I think having everything in contrib would
>> make it easier to resolve the disconnect between jsonb and hstore. As
>> things stand, there is a parallel set of functions and operators for
>> hstore and jsonb, with the former set much larger than the latter. I'm
>> not terribly happy with that.
>>
> The jsonb set will get larger as time goes on. I don't think either of you
> are thinking very clearly about how we would do this. Extensions can't call
> each other's code. So the whole notion we have here of sharing the tree-ish
> data representation and a lot of the C API would go out the window, unless
> you want to shoehorn jsonb into hstore. Frankly, we'll look silly with json
> as a core type and the more capable jsonb not.

It's not very clear to me why we think it's a good idea to share the
tree-ish representation between json and hstore. In deference to your
comments that this has been very publicly discussed over quite a
considerable period, I went back and tried to find the email in which
the drivers for that design decision were laid out. I can find no
such email; in fact, the first actual nested hstore patch I can find
is from January 13th and the first jsonb patch I can find is from
February 9th. Neither contains anything much more than the patch
itself, without anything at all describing the design, let alone
explaining why it was chosen. And although there are earlier mentions
of both nested hstore and jsonb, there's nothing that says, OK, this
is why we're doing it that way. Or if there is, I couldn't find it.

So I tried to tease it out from looking at the patches. As nearly as
I can tell, the reason for making jsonb use hstore's binary format is
because then we can build indexes on jsonbfield::hstore, and the
actual type conversion will be a no-op; and the reason for upgrading
hstore to allow nested keys is so that jsonb can map onto it. So from
where I sit this whole thing looks like a very complicated exercise to
try to reuse parts of the existing hstore opclasses until such time as
jsonb opclasses of its own. But if, as Josh postulates, those
opclasses are going to materialize within a matter of months, then the
whole need for these things to share the same binary format is going
to go away before 9.4 is even out the door. That may not be a good
enough reason to tie these things together inextricably. Once jsonb
has its own opclasses, it can ship as a standalone data type without
needing to depend on hstore or anything else.

I may well be missing some other benefit here, so please feel free to
enlighten me.

> Not to mention that if at this stage people suddenly decide we should change
> direction on a course that has been very publicly discussed over quite a
> considerable period, and for which Teodor and I and others have put in a
> great deal of work, I at least am going to be extremely annoyed (note the
> characteristic Australian used of massive understatement.)

Unless I've missed some emails sent earlier than the dates noted
above, which is possible, the comments by myself and others on this
thread ought to be regarded as timely review. The basic problem here
is that this patch wasn't timely submitted, still doesn't seem to be
very done, and it's getting rather late. We therefore face the usual
problem of deciding whether to commit something that we might regret
later. If jsonb turns out to the wrong solution to the json problem,
will there be community support for adding a jsonc type next year? I
bet not. You may think this is most definitely the right direction to
go and you may even be right, but our ability to maneuver and back out
of things goes down to nearly zero once a release goes out the door,
so I think it's entirely appropriate to question whether we're
charting the best possible course. But I certainly understand the
annoyance.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company