From: | Dilip Kumar <dilipbalaut(at)gmail(dot)com> |
---|---|
To: | Justin Pryzby <pryzby(at)telsasoft(dot)com> |
Cc: | Robert Haas <robertmhaas(at)gmail(dot)com>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Alexander Korotkov <a(dot)korotkov(at)postgrespro(dot)ru>, David Steele <david(at)pgmasters(dot)net>, Ildus Kurbangaliev <i(dot)kurbangaliev(at)gmail(dot)com>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
Subject: | Re: [HACKERS] Custom compression methods |
Date: | 2021-03-11 10:51:38 |
Message-ID: | CAFiTN-snEyq1QG5TgkoQeVyCvJHqw5m5Nq416ZpPcs372Z-4Gw@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Thu, Mar 11, 2021 at 10:07 AM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
>
> On Thu, Mar 11, 2021 at 8:50 AM Justin Pryzby <pryzby(at)telsasoft(dot)com> wrote:
> For INSERT SELECT we were already doing in the older version so we can
> include that code here, we will also have to include the patches for
> decompressing data before forming the composite types because without
> that we can not ensure that lz4 does not exist anywhere in the table.
> Said that with that also we can not ensure that it doesn't exist anywhere
> in the system because it might exist in the WAL and if you do the crash
> recovery then might get those lz4 compressed data back.
In updated patches, now INSERT INTO SELECT/VACUUM FULL/ CLUSTER will
re compress the data as per the latest compression method.
create table t(a text compression lz4);
insert into t values(repeat('a', 9000));
postgres[129360]=# select pg_column_compression(a) from t;
pg_column_compression
-----------------------
lz4
(1 row)
postgres[129360]=# alter TABLE t ALTER COLUMN a SET COMPRESSION pglz;
ALTER TABLE
postgres[129360]=# select pg_column_compression(a) from t;
pg_column_compression
-----------------------
lz4
(1 row)
postgres[129360]=# VACUUM FULL t;
VACUUM
postgres[129360]=# select pg_column_compression(a) from t;
pg_column_compression
-----------------------
pglz
(1 row)
IMHO, now we have a way for user to rewrite table using VACUUM
FULL/CLUSTER so I don't think we should force rewrite in ALTER SET
COMPRESSION.
In attached patch I am re compressing in INSERT INTO SELECT as well,
but honestly I think maybe we don't need to force that also on user
and we can only do this in VACUUM FULL or CLUSTER, thoughts?
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachment | Content-Type | Size |
---|---|---|
v34-0002-Expand-the-external-data-before-forming-the-tupl.patch | text/x-patch | 4.8 KB |
v34-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patch | text/x-patch | 37.0 KB |
v34-0005-Add-default_toast_compression-GUC.patch | text/x-patch | 11.4 KB |
v34-0003-Disallow-compressed-data-inside-container-types.patch | text/x-patch | 10.6 KB |
v34-0004-Built-in-compression-method.patch | text/x-patch | 109.9 KB |
v34-0006-Alter-table-set-compression.patch | text/x-patch | 20.2 KB |
v34-0007-default-to-with-lz4.patch | text/x-patch | 1.7 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | mickiewicz | 2021-03-11 11:29:49 | first add newly loaded plugin to the list then invoke _PG_init |
Previous Message | Daniel Gustafsson | 2021-03-11 10:41:22 | Re: OpenSSL 3.0.0 compatibility |