Re: Thoughts about NUM_BUFFER_PARTITIONS

From: wenhui qiu <qiuwenhuifx(at)gmail(dot)com>
To: Japin Li <japinli(at)hotmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, tomas(dot)vondra(at)enterprisedb(dot)com
Subject: Re: Thoughts about NUM_BUFFER_PARTITIONS
Date: 2024-02-20 02:16:00
Message-ID: CAGjGUAKuNr94bkU9Qi1f+Lrq26q1z22o4QjosjmqMOtBqGQX1Q@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Heikki Linnakangas
I saw git log found this commit:
https://github.com/postgres/postgres/commit/3acc10c997f916f6a741d0b4876126b7b08e3892
,I don't seem to see an email discussing this commit. As the commit log
tells us, we don't know exactly how large a value is optimal, and I believe
it's more flexible to make it as a parameter.Thank you very much
tomas.vondra for explaining the relationship, i see that MAX_SIMUL_LWLOCKS
was just doubled in this commit, is there a more appropriate ratio between
them?

```````````````````````````````````````````````````````````````````````````
commit 3acc10c997f916f6a741d0b4876126b7b08e3892
Author: Robert Haas <rhaas(at)postgresql(dot)org>
Date: Thu Oct 2 13:58:50 2014 -0400

Increase the number of buffer mapping partitions to 128.

Testing by Amit Kapila, Andres Freund, and myself, with and without
other patches that also aim to improve scalability, seems to indicate
that this change is a significant win over the current value and over
smaller values such as 64. It's not clear how high we can push this
value before it starts to have negative side-effects elsewhere, but
going this far looks OK.

`````````````````````````````````````````````````````````

wenhui qiu <qiuwenhuifx(at)gmail(dot)com> 于2024年2月20日周二 09:36写道:

> Hi Japlin Li
> Thank you for such important information ! Got it
>
> Japin Li <japinli(at)hotmail(dot)com> 于2024年2月19日周一 10:26写道:
>
>>
>> On Mon, 19 Feb 2024 at 00:56, Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>
>> wrote:
>> > On 2/18/24 03:30, Li Japin wrote:
>> >>
>> >> I find it seems need to change MAX_SIMUL_LWLOCKS if we enlarge the
>> NUM_BUFFER_PARTITIONS,
>> >> I didn’t find any comments to describe the relation between
>> MAX_SIMUL_LWLOCKS and
>> >> NUM_BUFFER_PARTITIONS, am I missing someghing?
>> >
>> > IMHO the relationship is pretty simple - MAX_SIMUL_LWLOCKS needs to be
>> > higher than NUM_BUFFER_PARTITIONS, so that the backend can acquire all
>> > the partition locks if needed.
>> >
>>
>> Thanks for the explanation! Got it.
>>
>> > There's other places that acquire a bunch of locks, and all of them need
>> > to be careful not to exceed MAX_SIMUL_LWLOCKS. For example gist has
>> > GIST_MAX_SPLIT_PAGES.
>> >
>> >
>> > regards
>>
>

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message David Rowley 2024-02-20 02:38:44 Support boolcol IS [NOT] UNKNOWN in partition pruning
Previous Message wenhui qiu 2024-02-20 01:36:45 Re: Thoughts about NUM_BUFFER_PARTITIONS