Re: BUG #18349: ERROR: invalid DSA memory alloc request size 1811939328, CONTEXT: parallel worker

On 21/2/2024 19:52, Tomas Vondra wrote:
> Hi,
>
> On 2/20/24 15:05, Alexey Ermakov wrote:
>> On 2024-02-16 20:40, Andrei Lepikhov wrote:
>>> Interesting. It correlates with one performance issue I have been
>>> trying to catch already 3 months. Could you provide some reproduction
>>> of that behavior?
>>>
>> Yes, I'm still trying to make reproducer, it will take some time. Thanks.
>>
> It's a bit weird it needs 1.8GB of memory, but perhaps that's also
> linked to the number of batches, somehow?
I found one possible weak point in the code of PHJ:
ExecParallelHashJoinSetUpBatches:

pstate->batches = dsa_allocate0(hashtable->area,
EstimateParallelHashJoinBatch(hashtable) * nbatch);

It could explain why we have such a huge memory allocation with a size
not bonded to a power of 2.
Also, it makes more sense that we already have a second report [1] with
the same value of DSA allocation.
So, can Alexey apply a trivial patch and check whether the bug has gone?

[1]
https://www.postgresql.org/message-id/flat/PAYP264MB3368AF4D63704E370B204C39FF582%40PAYP264MB3368.FRAP264.PROD.OUTLOOK.COM

--
regards,
Andrei Lepikhov
Postgres Professional