Re: Using per-transaction memory contexts for storing decoded tuples

On Sun, Oct 13, 2024 at 11:00 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Fri, Oct 11, 2024 at 3:40 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> >
> > Please find the attached patches.
> >

Thank you for reviewing the patch!

>
> @@ -343,9 +343,9 @@ ReorderBufferAllocate(void)
> */
> buffer->tup_context = GenerationContextCreate(new_ctx,
> "Tuples",
> - SLAB_LARGE_BLOCK_SIZE,
> - SLAB_LARGE_BLOCK_SIZE,
> - SLAB_LARGE_BLOCK_SIZE);
> + SLAB_DEFAULT_BLOCK_SIZE,
> + SLAB_DEFAULT_BLOCK_SIZE,
> + SLAB_DEFAULT_BLOCK_SIZE);
>
> Shouldn't we change the comment atop this change [1] which states that
> we should benchmark the existing values?

Agreed.

>
> One more thing we kept the max size as SLAB_DEFAULT_BLOCK_SIZE instead
> of something like we do with ALLOCSET_DEFAULT_SIZES, so we can
> probably write a comment as to why we choose to use the max_size same
> as init_size.

Agreed. I've updated the comment. Please review it.

> BTW, can we once try to use the max size as
> SLAB_LARGE_BLOCK_SIZE? Can it lead to the same problem with concurrent
> transactions where freeing larger blocks could be a problem, if so, we
> can at least write a comment for future reference.

I've tested with SLAB_LARGE_BLOCK_SIZE as the max size but it seems
that a huge memory fragmentation issue still happens. In the same
scenario I used before, the maximum amount of allocated memory during
logical decoding was 1.26GB with logical_decodiing_work_mem being
'256MB'.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com