From: | Andrey Borodin <x4mmm(at)yandex-team(dot)ru> |
---|---|
To: | Dilip Kumar <dilipbalaut(at)gmail(dot)com> |
Cc: | Robert Haas <robertmhaas(at)gmail(dot)com>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Alexander Korotkov <a(dot)korotkov(at)postgrespro(dot)ru>, David Steele <david(at)pgmasters(dot)net>, Ildus Kurbangaliev <i(dot)kurbangaliev(at)gmail(dot)com>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, PostgreSQL Developers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
Subject: | Re: [HACKERS] Custom compression methods |
Date: | 2020-12-29 17:48:49 |
Message-ID: | 323B1B01-DA42-419F-A99C-23E2C162D53B@yandex-team.ru |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
> 28 дек. 2020 г., в 11:14, Andrey Borodin <x4mmm(at)yandex-team(dot)ru> написал(а):
>
>> Thanks for the patch. Maybe we can allow setting custom compression
>> methods for wal compression as well.
>
> No, unfortunately, we can't use truly custom methods. Custom compression handlers are WAL-logged. So we can use only static set of hardcoded compression methods.
So, I've made some very basic benchmarks on my machine [0].
With pglz after checkpoint I observe 1146 and 1225 tps.
With lz4 I observe 1485 and 1524 tps.
Without wal_compression I see 1529 tps.
These observations can be explained with plain statement: pglz is bottleneck on my machine, lz4 is not.
While this effect can be reached with other means [1] I believe having lz4 for WAL FPIs would be much more CPU efficient.
FPA lz4 for WAL FPI patch v17. Changes: fixed some frontend issues, added some comments.
Best regards, Andrey Borodin.
[0] https://yadi.sk/d/6y5YiROXQRkoEw
[1] https://www.postgresql.org/message-id/flat/25991595-1848-4178-AA57-872B10309DA2%40yandex-team.ru#e7bb0e048358bcff281011dcf115ad42
Attachment | Content-Type | Size |
---|---|---|
v17-0007-Add-Lz4-compression-to-WAL-FPIs.patch | application/octet-stream | 11.9 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Paul Martinez | 2020-12-29 20:26:19 | [PATCH] Simplify permission checking logic in user.c |
Previous Message | Tom Lane | 2020-12-29 16:45:03 | Re: Let's start using setenv() |