Re: Improve compression speeds in pg_lzcompress.c

On 01/08/2013 10:19 AM, Takeshi Yamamuro wrote:
> Hi,
>
> (2013/01/07 22:36), Greg Stark wrote:
>> On Mon, Jan 7, 2013 at 10:21 AM, John R Pierce<pierce(at)hogranch(dot)com>
>> wrote:
>>> On 1/7/2013 2:05 AM, Andres Freund wrote:
>>>>
>>>> I think there should be enough bits available in the toast pointer to
>>>> indicate the type of compression. I seem to remember somebody even
>>>> posting a patch to that effect?
>>>> I agree that it's probably too late in the 9.3 cycle to start with
>>>> this.
>>>
>>>
>>> so an upgraded database would have old toasted values in the old
>>> compression
>>> format, and new toasted values in the new format in an existing table?
>>> that's kind of ugly.
>>
>> I haven't looked at the patch. It's not obvious to me from the
>> description that the output isn't backwards compatible. The way the LZ
>> toast compression works the output is self-describing. There are many
>> different outputs that would decompress to the same thing and the
>> compressing code can choose how hard to look for earlier matches and
>> when to just copy bytes wholesale but the decompression will work
>> regardless.
>
> My patch is not backwards compatible, so we need some features
> to switch these old and new disk formats.
Is it a feature of our compressed format that it is hard to make this
backwards compatible.

Only decompression should work anyway as we have not supported
physical compatibility in the other direction in our other tools.

That is, we don't have pg_downgrade :)

Hannu

>
> I think the discussion below is helpful in this use.
> That is, PGLZ_Header is used as this purpose.
> http://archives.postgresql.org/pgsql-hackers/2012-03/msg00971.php
>
> regards,