RE: Perform streaming logical transactions by background workers and parallel apply

On Wednesday, May 11, 2022 1:10 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Wed, May 11, 2022 at 9:35 AM Masahiko Sawada
> <sawada(dot)mshk(at)gmail(dot)com> wrote:
> >
> > On Tue, May 10, 2022 at 5:59 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
> wrote:
> > >
> > > On Tue, May 10, 2022 at 10:39 AM Masahiko Sawada
> <sawada(dot)mshk(at)gmail(dot)com> wrote:
> > > >
> > > > Having it optional seems a good idea. BTW can the user configure
> > > > how many apply bgworkers can be used per subscription or in the
> > > > whole system? Like max_sync_workers_per_subscription, is it better
> > > > to have a configuration parameter or a subscription option for
> > > > that? If so, setting it to 0 probably means to disable the parallel apply
> feature.
> > > >
> > >
> > > Yeah, that might be useful but we are already giving an option while
> > > creating a subscription whether to allow parallelism, so will it be
> > > useful to give one more way to disable this feature? OTOH, having
> > > something like max_parallel_apply_workers/max_bg_apply_workers at
> > > the system level can give better control for how much parallelism
> > > the user wishes to allow for apply work.
> >
> > Or we can have something like
> > max_parallel_apply_workers_per_subscription that controls how many
> > parallel apply workers can launch per subscription. That also gives
> > better control for the number of parallel apply workers.
> >
>
> I think we can go either way in this matter as both have their pros and cons. I
> feel limiting the parallel workers per subscription gives better control but
> OTOH, it may not allow max usage of parallelism because some quota from
> other subscriptions might remain unused. Let us see what Hou-San or others
> think on this matter?

Thanks for Amit and Sawada-san's comments !
I will think over these approaches and reply soon.

Best regards,
Hou zj