From: | Justin Pryzby <pryzby(at)telsasoft(dot)com> |
---|---|
To: | Andrey Borodin <x4mmm(at)yandex-team(dot)ru> |
Cc: | Michael Paquier <michael(at)paquier(dot)xyz>, pgsql-hackers(at)postgresql(dot)org, Thomas Munro <thomas(dot)munro(at)gmail(dot)com> |
Subject: | Re: Different compression methods for FPI |
Date: | 2021-03-15 18:09:18 |
Message-ID: | 20210315180918.GW29463@telsasoft.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Sun, Mar 14, 2021 at 07:31:35PM -0500, Justin Pryzby wrote:
> On Sat, Mar 13, 2021 at 08:48:33PM +0500, Andrey Borodin wrote:
> > > 13 марта 2021 г., в 06:28, Justin Pryzby <pryzby(at)telsasoft(dot)com> написал(а):
> > > Updated patch with a minor fix to configure.ac to avoid warnings on OSX.
> > > And 2ndary patches from another thread to allow passing recovery tests.
> > > Renamed to WAL_COMPRESSION_*
> > > Split LZ4 support to a separate patch and support zstd. These show that
> > > changes needed for a new compression method have been minimized, although not
> > > yet moved to a separate, abstracted compression/decompression function.
> >
> > Thanks! Awesome work!
> >
> > > These two patches are a prerequisite for this patch to progress:
> > > * Run 011_crash_recovery.pl with wal_level=minimal
> > > * Make sure published XIDs are persistent
> > >
> > > I don't know if anyone will consider this patch for v14 - if not, it should be
> > > set to v15 and revisit in a month.
> >
> > I want to note, that fixes for 011_crash_recovery.pl are not strictly necessary for this patch set.
> > The problem in tests arises if we turn on wal_compression, absolutely independently from wal compression method.
> > We turn on wal_compression in this test only for CI purposes.
>
> I rearranged the patches to reflect this.
> Change to zlib and zstd to level=1.
> Add support for negative "zstd fast" levels.
> Use correct length accounting for "hole" in LZ4 and ZSTD.
> Fixed Solution.pm for zstd on windows.
> Switch to zstd by default (for CI).
> Add docs.
Changes:
- Allocate buffer sufficient to accommodate any supported compression method;
- Use existing info flags argument rather than adding another byte for storing
the compression method; this seems to be what was anticipated by commit
57aa5b2bb and what Michael objected to.
I think the existing structures are ugly, so maybe this suggests using a GUC
assign hook to support arbitrary compression level, and maybe other options.
--
Justin
Attachment | Content-Type | Size |
---|---|---|
0001-Allow-alternate-compression-methods-for-wal_compress.patch | text/x-diff | 15.0 KB |
0002-Run-011_crash_recovery.pl-with-wal_level-minimal.patch | text/x-diff | 1001 bytes |
0003-Make-sure-published-XIDs-are-persistent.patch | text/x-diff | 7.2 KB |
0004-wal_compression_method-default-to-zlib.patch | text/x-diff | 1.5 KB |
0005-re-add-wal_compression_method-lz4.patch | text/x-diff | 13.6 KB |
0006-Default-to-LZ4.patch | text/x-diff | 2.8 KB |
0007-add-wal_compression_method-zstd.patch | text/x-diff | 13.6 KB |
0008-Default-to-zstd.patch | text/x-diff | 2.7 KB |
0009-Add-zstd-compression-levels.patch | text/x-diff | 4.5 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Julien Rouhaud | 2021-03-15 18:10:37 | Re: REINDEX backend filtering |
Previous Message | Pavel Stehule | 2021-03-15 18:02:39 | Re: Parser Hook |