Re: suboverflowed subtransactions concurrency performance optimize

From: Andrey Borodin <x4mmm(at)yandex-team(dot)ru>
To: Pengchengliu <pengchengliu(at)tju(dot)edu(dot)cn>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: suboverflowed subtransactions concurrency performance optimize
Date: 2021-08-30 10:24:57
Message-ID: EE4E9BF3-A824-4940-B13F-7448B43DA500@yandex-team.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Pengcheng!

You are solving important problem, thank you!

> 30 авг. 2021 г., в 13:43, Pengchengliu <pengchengliu(at)tju(dot)edu(dot)cn> написал(а):
>
> To resolve this performance problem, we think about a solution which cache
> SubtransSLRU to local cache.
> First we can query parent transaction id from SubtransSLRU, and copy the
> SLRU page to local cache page.
> After that if we need query parent transaction id again, we can query it
> from local cache directly.

A copy of SLRU in each backend's cache can consume a lot of memory. Why create a copy if we can optimise shared representation of SLRU?

JFYI There is a related patch to make SimpleLruReadPage_ReadOnly() faster for bigger SLRU buffers[0].
Also Nik Samokhvalov recently published interesting investigation on the topic, but for some reason his message did not pass the moderation. [1]

Also it's important to note that there was a community request to move SLRUs to shared_buffers [2].

Thanks!

Best regards, Andrey Borodin.

[0] https://commitfest.postgresql.org/34/2627/
[1] https://www.postgresql.org/message-id/flat/BE73A0BB-5929-40F4-BAF8-55323DE39561%40yandex-team.ru
[2] https://www.postgresql.org/message-id/flat/20180814213500.GA74618%4060f81dc409fc.ant.amazon.com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Bharath Rupireddy 2021-08-30 10:31:24 Re: Trap errors from streaming child in pg_basebackup to exit early
Previous Message Kasahara Tatsuhito 2021-08-30 10:16:09 Re: Error code for checksum failure in origin.c