Re: Avoid detoast overhead when possible

Hi!

Sorry for the long delay, other tasks required my attention and this project
was postponed, but we have some recent requests from clients and this
work has to be revived.

1) In short the idea behind the Json improvements is to introduce iterators
for Json objects to extract these objects value by value, and modification
of in-memory and on-disk representations to hold partially-detoasted Json
objects (tree nodes with need-detoast state) for in-memory, and to store
key-value map in on-disk representation to extract only required values.

2) Backward compatibility would not be a problem because in on-disk
representation we could distinguish old version from new, there are enough
service bits for that. But servers not updated with this patch could not
parse new data, of course.

3) Yes, it definitely has to. These changes are very complex and invasive,
that's the reason I haven't finished the patch.

4) It doesn't seem worthy at first glance. Do you have any ideas on this?

Thank you for your interest!

On Wed, Oct 30, 2024 at 2:59 AM Andy Fan <zhihuifan1213(at)163(dot)com> wrote:

> zhihuifan1213(at)163(dot)com writes:
>
> Hello Nikita,
>
> >> There's a view from the other angle - detoast just attributes that are
> needed
> >> (partial detoast), with optimized storage mechanics for JSONb.
> >
> > Very glad to know that, looking forward your design & patch!
>
> This is interesting, do you mind to provide a high level design of
> this. I'm not pushing to do this, but you asked me how I think about
> your proposal[1], so I think a high level design is a must to for these
> question. A first set of questions includes:
>
> 1. How to optimize the JSONb storage?
> 2. How to be compatible with older version?
> 3. How to integreate with existing ExprExecutionEngine? looks your
> proposal needs to know which "part" of jsonb fields are needed.
> 4. Is there any optimization if user want to get a full JSONB, like
> SELECT jb_col FROM t?
>
> [1]
>
> https://www.postgresql.org/message-id/CAN-LCVO3GZAKVTKNwwcezoc%3D9Lq%3DkU2via-BM3MXVdOq4tD9RQ%40mail.gmail.com
>
> --
> Best Regards
> Andy Fan
>
>

--
Regards,
Nikita Malakhov
Postgres Professional
The Russian Postgres Company
https://postgrespro.ru/