From: | Haribabu Kommi <kommi(dot)haribabu(at)gmail(dot)com> |
---|---|
To: | Fujii Masao <masao(dot)fujii(at)gmail(dot)com> |
Cc: | Simon Riggs <simon(at)2ndquadrant(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: Compression of full-page-writes |
Date: | 2014-05-13 00:34:45 |
Message-ID: | CAJrrPGfxGg9FyvwLLMu=k2bF=7TJ+53ji+p-QnUuOyBF6kJ3oA@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Tue, May 13, 2014 at 3:33 AM, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
> On Sun, May 11, 2014 at 7:30 PM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
>> On 30 August 2013 04:55, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
>>
>>> My idea is very simple, just compress FPW because FPW is
>>> a big part of WAL. I used pglz_compress() as a compression method,
>>> but you might think that other method is better. We can add
>>> something like FPW-compression-hook for that later. The patch
>>> adds new GUC parameter, but I'm thinking to merge it to full_page_writes
>>> parameter to avoid increasing the number of GUC. That is,
>>> I'm thinking to change full_page_writes so that it can accept new value
>>> 'compress'.
>>
>>> * Result
>>> [tps]
>>> 1386.8 (compress_backup_block = off)
>>> 1627.7 (compress_backup_block = on)
>>>
>>> [the amount of WAL generated during running pgbench]
>>> 4302 MB (compress_backup_block = off)
>>> 1521 MB (compress_backup_block = on)
>>
>> Compressing FPWs definitely makes sense for bulk actions.
>>
>> I'm worried that the loss of performance occurs by greatly elongating
>> transaction response times immediately after a checkpoint, which were
>> already a problem. I'd be interested to look at the response time
>> curves there.
>
> Yep, I agree that we should check how the compression of FPW affects
> the response time, especially just after checkpoint starts.
>
>> I was thinking about this and about our previous thoughts about double
>> buffering. FPWs are made in foreground, so will always slow down
>> transaction rates. If we could move to double buffering we could avoid
>> FPWs altogether. Thoughts?
>
> If I understand the double buffering correctly, it would eliminate the need for
> FPW. But I'm not sure how easy we can implement the double buffering.
There is already a patch on the double buffer write to eliminate the FPW.
But It has some performance problem because of CRC calculation for the
entire page.
I think this patch can be further modified with a latest multi core
CRC calculation and can be used for testing.
Regards,
Hari Babu
Fujitsu Australia
From | Date | Subject | |
---|---|---|---|
Next Message | Robert Haas | 2014-05-13 01:00:57 | Re: New timezones used in regression tests |
Previous Message | Tom Lane | 2014-05-12 23:32:51 | Re: pgsql: Revive line type |