Re: Avoid detoast overhead when possible

Nikita Malakhov <hukutoc(at)gmail(dot)com> writes:

> 1) In short the idea behind the Json improvements is to introduce iterators
> for Json objects to extract these objects value by value,

OK here.

> and modification
> of in-memory and on-disk representations to hold partially-detoasted Json
> objects (tree nodes with need-detoast state) for in-memory, and to store
> key-value map in on-disk representation to extract only required
> values.

I have troubles to follow here. Could you explain this more for the
below example, What will be done differently from current in the below
example:

CREATE TABLE t(jbcol jsonb);

INSERT INTO t values ('{"a": 1, "b": 2}'); // new storage format?

SELECT jbcol->'a', jbcol->'b' FROM t; // How does ExprInterp.c work
with this?

> 2) Backward compatibility would not be a problem because in on-disk
> representation we could distinguish old version from new.

OK, so you indeed change the storage format, if so, How is the new
format looks like? It would be an important factor to discuss.

> there are enough service bits for that.

OK, good news.

> But servers not updated with this patch could not parse new data, of
> course.

This looks too for me.

>
> 3) Yes, it definitely has to. These changes are very complex and invasive,
> that's the reason I haven't finished the patch.

This is the area I am really worried about. I think you can have some
high-level design document for review first. It would be great that some
more experienced people could have a look at, however I know it is hard
becuase of their bindwidth. You need that document anyway.

Without the details, other people is hard to understand the idea in
your mind.

>
> 4) It doesn't seem worthy at first glance. Do you have any ideas on
> this?

I think the more important ones are (a) what the new storage looks like,
(b) how does it works with the ExprExecutionEnginner. Without that
infromation, I think it is too soon to talk about this.

--
Best Regards
Andy Fan