Re: scalability bottlenecks with (many) partitions (and more)

On 9/4/24 17:12, David Rowley wrote:
> On Wed, 4 Sept 2024 at 03:06, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>>
>> On Mon, Sep 2, 2024 at 1:46 PM Tomas Vondra <tomas(at)vondra(dot)me> wrote:
>>> But say we add a GUC and set it to -1 by default, in which case it just
>>> inherits the max_locks_per_transaction value. And then also provide some
>>> basic metric about this fast-path cache, so that people can tune this?
>>
>> All things being equal, I would prefer not to add another GUC for
>> this, but we might need it.
>
> I think driving the array size from max_locks_per_transaction is a
> good idea (rounded up to the next multiple of 16?).

Maybe, although I was thinking we'd just use the regular doubling, to
get nice "round" numbers. It will likely overshoot a little bit (unless
people set the GUC to exactly 2^N), but I don't think that's a problem.

> If someone comes along one day and shows us a compelling case where
> some backend needs more than its fair share of locks and performance
> is bad because of that, then maybe we can consider adding a GUC then.
> Certainly, it's much easier to add a GUC later if someone convinces
> us that it's a good idea than it is to add it now and try to take it
> away in the future if we realise it's not useful enough to keep.
>

Yeah, I agree with this.

regards

--
Tomas Vondra