Re: Reference to - BUG #18349: ERROR: invalid DSA memory alloc request size 1811939328, CONTEXT: parallel worker

On 18/10/2024 06:34, Thomas Munro wrote:
> On Thu, Oct 17, 2024 at 10:48 PM Andrei Lepikhov <lepihov(at)gmail(dot)com> wrote:
>> BTW, why not to use current case and fix the problem with the 'invalid
>> DSA memory alloc request size 1811939328' itself ?
>
> I think your patch is good but if you don't mind I'd like to think
> about how to generalise it a bit first, so that it applies to all
> places where we choose nbatch, not just repartitioning. Unfortunately
> that's a bit circular so I'm still thinking about the tidiest way to
> do it... might take a few days due to travel, and if I don't have
> something soon I guess your patch is better than nothing (it might be
> the most common way we finish up in that sort of trouble).
Thanks. I thought about generalisation, but who knows how the caller
wants to react in the case of unsuccessful allocation? Should we force
the user to compare the size requested and allocated? Maybe. I'll wait
for your solution.
>
> I'll also push that other patch that cleans up temporary files
> aggressively soon (master only),
Have been waiting for this!
> and try to think about some simple
> ways to avoid large nbatch values that contradict the goal of reducing
> memory for non-skew cases at planning and execution time (probably
> master only)...
Hmm, cost-based maximum number of batches defined at the optimisation stage?

--
regards, Andrei Lepikhov