Re: pg_dump -Fd and compression level

On 07/24/2015 05:22 PM, Marc Mamin wrote:
>>> After our last upgrade, we've noticed a 10-20% size increase of our dump size.
>>> This comes from our backup scripts were pg_dump was called without setting -Z
>>>
>>> So it seems, that this fix did modify the default compression to use:
>>> http://michael.otacoo.com/postgresql-2/pg_dump-directory-format-compression/
>>>
>>> not sure if this is expected or if this commit accidently changed the default compression, setting it too low.
>>>
>>> moreover, the doc is somewhat unclear here as it mentions all formats but the directory one:
>>>
>>> -Z 0..9
>>> --compress=0..9
>>>
>>> Specify the compression level to use. Zero means no compression.
>>> For the custom archive format, this specifies compression of individual
>>> table-data segments, and the default is to compress at a moderate level.
>>> For plain text output, setting a nonzero compression level causes the entire
>>> output file to be compressed, as though it had been fed through gzip;
>>> but the default is not to compress.
>>> The tar archive format currently does not support compression at all.
>>>
>>> shouldn't that be changed to
>>>
>>> - For the custom archive format
>>> + For the directory and custom archive formats
>>>
>>>
>> What did you upgrade from/to?
> 9.3.6 to 9.3.9
>
> this is bound to this 9.3.7 fix:
> "In pg_dump, fix failure to honor -Z compression level option together with -Fd (Michael Paquier)"
>
> I understand that the modification is wishfull, but the change has nevertheless a non negligable impact.
> This had increased our backup repository of about 1TB within a few weeks if we hadn't noticed it.
>
>

Hmm. Yeah. It looks like commit a7ad5cf0cfcfab8418000d652fa4f0c6ad6c8911
changed from using the default compression for libz to using the
compression set in pg_dump options, which defaults to 0. This actually
seems like the right thing to do, but it certainly should have been
called out much more forcefully in release notes, and arguably should
not have been changed in stable releases. Not sure what we do about it now.

cheers

andrew