Re: Wanted: jsonb on-disk representation documentation

Hi,

On 2014-05-07 14:20:19 +0300, Heikki Linnakangas wrote:
> So, apart from cleaning up the code, we really need to take a close look at
> the on-disk format now. The code can be cleaned up later, too, but we're
> going to be stuck with the on-disk format forever, so it's critical to get
> that right.

+1

> First, a few observations:

Agreed.

I'd like to add that:
* Imo we need space in jsonb ondisk values to indicate a format
version. We won't fully get this right.
* The jentry representation should be changed so it's possible to get the type
of a entry without checking individual types. That'll make code like
findJsonbValueFromSuperHeader() (well, whatever you've renamed it to)
much easier to read. Preferrably so it an have the same values (after
shifting/masking) ask the JBE variants. And it needs space for futher
types (or representations thereof).
* I wonder if the hash/object pair representation is wise and if it
shouldn't be keys combined with offsets first, and then the
values. That will make access much faster. And that's what jsonb
essentially is about.
* I think both arrays and hashes should grow individual structs. With
space for additional flags.

* I have doubts of the wisdom of allowing to embed jbvBinary values in
JsonbValues. Although that can be changed later since it's not on disk.

> I'm going to proceed refactoring those things, which will change the on-disk
> format. It's late in the release cycle - these things really should've been
> cleaned up earlier - but it's important to get the on-disk format right.
> Shout if you have any objections.

I don't think it's likely that beta1 will be binary compatible with the
final version at this point.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services