Re: Improve eviction algorithm in ReorderBuffer

On Wed, Dec 20, 2023 at 6:49 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>
> On Tue, Dec 19, 2023 at 8:02 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > On Tue, Dec 19, 2023 at 8:31 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> > >
> > > On Sun, Dec 17, 2023 at 11:40 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> > > >
> > > >
> > > > The individual transactions shouldn't cross
> > > > 'logical_decoding_work_mem'. I got a bit confused by your proposal to
> > > > maintain the lists: "...splitting it into two lists: transactions
> > > > consuming 5% < and 5% >= of the memory limit, and checking the 5% >=
> > > > list preferably.". In the previous sentence, what did you mean by
> > > > transactions consuming 5% >= of the memory limit? I got the impression
> > > > that you are saying to maintain them in a separate transaction list
> > > > which doesn't seems to be the case.
> > >
> > > I wanted to mean that there are three lists in total: the first one
> > > maintain the transactions consuming more than 10% of
> > > logical_decoding_work_mem,
> > >
> >
> > How can we have multiple transactions in the list consuming more than
> > 10% of logical_decoding_work_mem? Shouldn't we perform serialization
> > before any xact reaches logical_decoding_work_mem?
>
> Well, suppose logical_decoding_work_mem is set to 64MB, transactions
> consuming more than 6.4MB are added to the list. So for example, it's
> possible that the list has three transactions each of which are
> consuming 10MB while the total memory usage in the reorderbuffer is
> still 30MB (less than logical_decoding_work_mem).
>

Thanks for the clarification. I misunderstood the list to have
transactions greater than 70.4 MB (64 + 6.4) in your example. But one
thing to note is that maintaining these lists by default can also have
some overhead unless the list of open transactions crosses a certain
threshold.

--
With Regards,
Amit Kapila.