Re: Force re-compression with lz4

From: Mladen Gogala <gogala(dot)mladen(at)gmail(dot)com>
To: pgsql-general(at)lists(dot)postgresql(dot)org
Subject: Re: Force re-compression with lz4
Date: 2021-10-18 13:41:33
Message-ID: d4ff3c0b-ac43-6b00-622a-501870fc2810@gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general


On 10/18/21 01:07, Michael Paquier wrote:
> CPU-speaking, LZ4 is*much* faster than pglz when it comes to
> compression or decompression with its default options. The
> compression ratio is comparable between both, still LZ4 compresses in
> average less than PGLZ.
> --
> Michael

LZ4 works much better with deduplication tools like Data Domain or Data
Domain Boost (client side deduplication). With zip or gzip compression,
deduplication ratios are much lower than with LZ4. Most of the modern
backup tools (DD, Veeam, Rubrik, Commvault) support deduplication. LZ4
algorithm uses less CPU than zip, gzip or bzip2 and works much better
with deduplication algorithms employed by the backup tools. This is
actually a very big and positive change.

Disclosure:

I used to work for Commvault as a senior PS engineer. Commvault was the
first tool on the market to combine LZ4 and deduplication.

Regards

--
Mladen Gogala
Database Consultant
Tel: (347) 321-1217
https://dbwhisperer.wordpress.com

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Tom Lane 2021-10-18 13:45:19 Re: PostgreSQL 14: pg_dump / pg_restore error: could not write to the communication channel: Broken pipe
Previous Message Nick Renders 2021-10-18 11:42:22 Re: PostgreSQL 14: pg_dump / pg_restore error: could not write to the communication channel: Broken pipe