Re: ZStandard (with dictionaries) compression support for TOAST compression

From: Nikita Malakhov <hukutoc(at)gmail(dot)com>
To: Nikhil Kumar Veldanda <veldanda(dot)nikhilkumar17(at)gmail(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Michael Paquier <michael(at)paquier(dot)xyz>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: ZStandard (with dictionaries) compression support for TOAST compression
Date: 2025-04-29 10:36:51
Message-ID: CAN-LCVMsmtxnEycec3SRxvCmwgbYTSkbF477Bsj7xT1zobwPig@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

Nikhil, please consider existing discussions on using dictionaries
(mentioned above by Aleksander) and extending the TOAST pointer [1],
it seems you did not check them.

The same question Robert asked above - it's unclear why the header
wastes so much space. You mentioned metadata length - what metadata
do you mean there?

Also Robert pointed out very questionable approaches in your solution -
new dependencies crawling around user tables, new catalog table
with very unclear lifecycle (and, having new catalog table, immediately
having questions with pg_upgrade).
Currently I'm looking through the patch and could share my thoughts
later.

While reading this thread I've thought about storing a dictionary within
the table it is used for - IIUC on dictionary is used for just one
attribute,
so it does not make sense to make it global.

Also, I have a question regarding the Zstd implementation you propose -
does it provide a possibility for partial decompression?

>
>
Thanks!

[1]
https://www.postgresql.org/message-id/flat/CAN-LCVMq2X%3Dfhx7KLxfeDyb3P%2BBXuCkHC0g%3D9GF%2BJD4izfVa0Q%40mail.gmail.com

--
Regards,
Nikita Malakhov
Postgres Professional
The Russian Postgres Company
https://postgrespro.ru/

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message jian he 2025-04-29 10:39:42 add tab-complete for ALTER DOMAIN ADD...
Previous Message Alexander Korotkov 2025-04-29 10:00:05 Re: type cache cleanup improvements