RE: [Patch] Optimize dropping of relation buffers using dlist

From: "k(dot)jamison(at)fujitsu(dot)com" <k(dot)jamison(at)fujitsu(dot)com>
To: 'Amit Kapila' <amit(dot)kapila16(at)gmail(dot)com>
Cc: Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, "tsunakawa(dot)takay(at)fujitsu(dot)com" <tsunakawa(dot)takay(at)fujitsu(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: RE: [Patch] Optimize dropping of relation buffers using dlist
Date: 2020-12-14 03:00:12
Message-ID: OSBPR01MB2341882F416A282C3F7D769DEFC70@OSBPR01MB2341.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Friday, December 11, 2020 10:27 AM, Amit Kapila wrote:
> On Fri, Dec 11, 2020 at 5:54 AM k(dot)jamison(at)fujitsu(dot)com
> <k(dot)jamison(at)fujitsu(dot)com> wrote:
> > So should I still not include that information?
> >
>
> I think we can extend your existing comment like: "Otherwise if the size of a
> relation fork is not cached, we proceed to a full scan of the whole buffer pool.
> This can happen if there is no update to a particular fork during recovery."

Attached are the final updated patches.
I followed this advice and updated the source code comment a little bit.
There are no changes from the previous except that and the unnecessary
"cached" condition which Tsunakawa-san mentioned.

Below is also the updated recovery performance test results for TRUNCATE.
(1000 tables, 1MB per table, results measured in seconds)
| s_b | Master | Patched | % Reg |
|-------|--------|---------|---------|
| 128MB | 0.406 | 0.406 | 0% |
| 512MB | 0.506 | 0.406 | -25% |
| 1GB | 0.806 | 0.406 | -99% |
| 20GB | 15.224 | 0.406 | -3650% |
| 100GB | 81.506 | 0.406 | -19975% |

Because of the size of relation, it is expected to enter full-scan for
the 128MB shared_buffers setting. And there was no regression.
Similar to previous test results, the recovery time was constant
for all shared_buffers setting with the patches applied.

Regards,
Kirk Jamison

Attachment Content-Type Size
v35-0001-Prevent-invalidating-blocks-in-smgrextend-during.patch application/octet-stream 1.1 KB
v35-0002-Add-bool-param-in-smgrnblocks-for-cached-blocks.patch application/octet-stream 9.5 KB
v35-0003-Optimize-DropRelFileNodeBuffers-during-recovery.patch application/octet-stream 9.3 KB
v35-0004-Optimize-DropRelFileNodesAllBuffers-in-recovery.patch application/octet-stream 6.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2020-12-14 03:00:37 Re: pg_waldump error message fix
Previous Message Kyotaro Horiguchi 2020-12-14 02:34:51 Re: pg_waldump error message fix