Re: Perform streaming logical transactions by background workers and parallel apply

On Tue, May 10, 2022 at 10:39 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>
> On Wed, May 4, 2022 at 8:44 AM Peter Smith <smithpb2250(at)gmail(dot)com> wrote:
> >
> > On Tue, May 3, 2022 at 5:16 PM Peter Smith <smithpb2250(at)gmail(dot)com> wrote:
> > >
> > ...
> >
> > > Avoiding unexpected differences like this is why I suggested the
> > > option should have to be explicitly enabled instead of being on by
> > > default as it is in the current patch. See my review comment #14 [1].
> > > It means the user won't have to change their existing code as a
> > > workaround.
> > >
> > > ------
> > > [1] https://www.postgresql.org/message-id/CAHut%2BPuqYP5eD5wcSCtk%3Da6KuMjat2UCzqyGoE7sieCaBsVskQ%40mail.gmail.com
> > >
> >
> > Sorry I was wrong above. It seems this behaviour was already changed
> > in the latest patch v5 so now the option value 'on' means what it
> > always did. Thanks!
>
> Having it optional seems a good idea. BTW can the user configure how
> many apply bgworkers can be used per subscription or in the whole
> system? Like max_sync_workers_per_subscription, is it better to have a
> configuration parameter or a subscription option for that? If so,
> setting it to 0 probably means to disable the parallel apply feature.
>

Yeah, that might be useful but we are already giving an option while
creating a subscription whether to allow parallelism, so will it be
useful to give one more way to disable this feature? OTOH, having
something like max_parallel_apply_workers/max_bg_apply_workers at the
system level can give better control for how much parallelism the user
wishes to allow for apply work. If we have such a new parameter then I
think max_logical_replication_workers should include apply workers,
parallel apply workers, and table synchronization? In such a case,
don't we need to think of increasing the default value of
max_logical_replication_workers?

--
With Regards,
Amit Kapila.