Re: maintenance_work_mem used by Vacuum

On Thu, Oct 17, 2019 at 6:05 PM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>
> On Thu, Oct 17, 2019 at 6:13 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > On Wed, Oct 16, 2019 at 5:35 PM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> > >
> > > On Wed, Oct 16, 2019 at 3:48 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> > > >
> > > >
> > > > It is not that currently, other indexes don't use any additional
> > > > memory (except for maintainence_work_mem). For example, Gist index
> > > > can use memory for collecting empty leaf pages and internal pages. I
> > > > am not sure if we can do anything for such cases. The case for Gin
> > > > index seems to be clear and it seems to be having the risk of using
> > > > much more memory, so why not try to do something for it?
> > >
> > > Yeah gin indexes is clear now and I agree that we need to do something
> > > for it. But I'm also concerned third party index AMs. Similar to the
> > > problem related to IndexBulkDeleteResult structure that we're
> > > discussing on another thread I thought that we have the same problem
> > > on this.
> > >
> >
> > I understand your concern, but I am not sure what is a good way to
> > deal with it. I think we can do something generic like divide the
> > maintainence_work_mem equally among workers, but then the indexes that
> > use maintainence_work_mem will suffer if the number of such indexes is
> > much less than the indexes that don't use maintainence_work_mem.
> > Another idea could be each index AM tell whether it uses
> > maintainence_work_mem or not and based on that we can do the
> > computation (divide the maintainence_work_mem by the number of such
> > indexes during parallel vacuum). Do you have any other ideas for
> > this?
> >
>
> I was thinking the similar idea to the latter idea: ask index AM the
> amount of memory (that should be part of maintenance_work_mem) it will
> consume and then compute the new limit for both heap scan and index
> vacuuming based on that.
>

Oh, that would be tricky as compared to what I am proposing because it
might not be easy for indexAM to tell upfront the amount of memory it
needs. I think we can keep it simple for now.

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com