Re: [PATCH] Compression dictionaries for JSONB

Hi!

If I understand Andres' message correctly - the proposition is to
make use of compression dictionaries automatic, possibly just setting
a parameter when the table is created, something like
CREATE TABLE t ( ..., t JSONB USE DICTIONARY);
The question is then how to create such dictionaries automatically
and extend them while data is being added to the table. Because
it is not something unusual when after a time circumstances change
and a rather small table is started to be loaded with huge amounts
of data.

I prefer extending a dictionary over re-creating it because while
dictionary is recreated we leave users two choices - to wait until
dictionary creation is over or to use the old version (say, kept as
as a snapshot while a new one is created). Keeping many versions
simultaneously does not make sense and would extend DB size.

Also, compressing small data with a large dictionary (the case for
one-for-many tables dictionary), I think, would add some considerable
overhead to the INSERT/UPDATE commands, so the most reasonable
choice is a per-table dictionary.

Am I right?

Any ideas on how to create and extend such dictionaries automatically?

On Thu, Feb 9, 2023 at 2:01 PM Andres Freund <andres(at)anarazel(dot)de> wrote:

> Hi,
>
> On February 9, 2023 2:50:57 AM PST, Aleksander Alekseev <
> aleksander(at)timescale(dot)com> wrote:
> >Hi Andres,
> >
> >> > So to clarify, are we talking about tuple-level compression? Or
> >> > perhaps page-level compression?
> >>
> >> Tuple level.
> >
> >> although my own patch proposed attribute-level compression, not
> >> tuple-level one, it is arguably closer to tuple-level approach than
> >> page-level one
> >
> >Just wanted to make sure that by tuple-level we mean the same thing.
> >
> >When saying tuple-level do you mean that the entire tuple should be
> >compressed as one large binary (i.e. similarly to page-level
> >compression but more granularly), or every single attribute should be
> >compressed separately (similarly to how TOAST does this)?
>
> Good point - should have been clearer. I meant attribute wise compression.
> Like we do today, except that we would use a dictionary to increase
> compression rates.
>
> Andres
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>

--
Regards,
Nikita Malakhov
Postgres Professional
https://postgrespro.ru/