Re: Map WAL segment files on PMEM as WAL buffers

From: Takashi Menjo <takashi(dot)menjo(at)gmail(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Cc: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, tsunakawa(dot)takay(at)fujitsu(dot)com, "Deng, Gang" <gang(dot)deng(at)intel(dot)com>, Takashi Menjo <takashi(dot)menjou(dot)vg(at)hco(dot)ntt(dot)co(dot)jp>
Subject: Re: Map WAL segment files on PMEM as WAL buffers
Date: 2021-06-18 06:44:54
Message-ID: CAOwnP3PWp=bDp4CJoaNn6F9KoYKKZDWs88y9eThxS92PcZ4WpA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi hackers,

The v2 patchset, an updated performance report, and a zipped
supplemental patchset for comparison in the report. I confirmed that
the v2 can be applied to the latest master d24c565 (Jun 17 2021)
without conflict, but the supplemental patchset cannot. If you want to
reproduce the comparison, please apply to eb43bdb (May 25, 2021) as I
did so in the report.

The v2 includes WAL statistics support and WAL pre-allocation feature
in cases of PMEM, and some fixes for the first version. The size of
WAL buffers managed by xlblocks becomes min_wal_size, and the buffers
and underlying segment files are initialized at startup then
periodically by walwriter. This looks to improve performance, as I
wrote in the report.

By the way, I found that Nathan-san posted a PoC patch for WAL
pre-allocation in another thread [1]. I will pay attention to it and
discussions related to WAL pre-allocation in pgsql-hackers.

Best regards,
Takashi

[1] https://www.postgresql.org/message-id/flat/20201225200953.jjkrytlrzojbndh5%40alap3.anarazel.de

--
Takashi Menjo <takashi(dot)menjo(at)gmail(dot)com>

Attachment Content-Type Size
v2-0005-WAL-statistics-in-cases-of-wal_pmem_map-true.patch application/octet-stream 2.2 KB
v2-0004-Map-WAL-segment-files-on-PMEM-as-WAL-buffers.patch application/octet-stream 17.0 KB
v2-0001-Add-with-libpmem-option-for-PMEM-support.patch application/octet-stream 5.3 KB
v2-0002-Add-wal_pmem_map-to-GUC.patch application/octet-stream 4.7 KB
v2-0003-Export-InstallXLogFileSegment.patch application/octet-stream 2.1 KB
v2-0006-Preallocate-and-initialize-more-WAL-if-wal_pmem_m.patch application/octet-stream 3.6 KB
Performance-of-PMEM-patchsets-20210618.pdf application/pdf 1.7 MB
Non-volatile-WAL-buffer-20210525.zip application/x-zip-compressed 37.5 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2021-06-18 06:59:32 Re: Refactor ECPGconnect and allow IPv6 connection
Previous Message Michael Paquier 2021-06-18 06:43:22 Re: Toast compression method options