Re: Using per-transaction memory contexts for storing decoded tuples

On 2024/09/19 8:53, Masahiko Sawada wrote:
> On Tue, Sep 17, 2024 at 2:06 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>>
>> On Mon, Sep 16, 2024 at 10:43 PM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>>>
>>> On Fri, Sep 13, 2024 at 3:58 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>>>>
>>>> Can we try reducing the size of
>>>> 8MB memory blocks? The comment atop allocation says: "XXX the
>>>> allocation sizes used below pre-date generation context's block
>>>> growing code. These values should likely be benchmarked and set to
>>>> more suitable values.", so do we need some tuning here?
>>>
>>> Reducing the size of the 8MB memory block would be one solution and
>>> could be better as it could be back-patchable. It would mitigate the
>>> problem but would not resolve it. I agree to try reducing it and do
>>> some benchmark tests. If it reasonably makes the problem less likely
>>> to happen, it would be a good solution.
>>>
>>
>> makes sense.
>
> I've done some benchmark tests for three different code bases with
> different test cases. In short, reducing the generation memory context
> block size to 8kB seems to be promising; it mitigates the problem
> while keeping a similar performance.

Sounds good!

I believe the memory bloat issue in the reorder buffer should be
considered a bug. Since this solution isn’t too invasive, I think
it’s worth considering back-patch to older versions.

Then, if we find a better approach, we can apply it to v18 or later.

Regards,

--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION