Re: Perform streaming logical transactions by background workers and parallel apply

On Thu, Dec 8, 2022 at 1:52 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Wed, Dec 7, 2022 at 6:33 PM houzj(dot)fnst(at)fujitsu(dot)com
> <houzj(dot)fnst(at)fujitsu(dot)com> wrote:
> >
> > On Wednesday, December 7, 2022 7:51 PM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> > >
> >
> > > ---
> > > When max_parallel_apply_workers_per_subscription is changed to a value
> > > lower than the number of parallel worker running at that time, do we
> > > need to stop extra workers?
> >
> > I think we can do this, like adding a check in the main loop of leader worker, and
> > check every time after reloading the conf. OTOH, we will also stop the worker after
> > finishing a transaction, so I am slightly not sure do we need to add another check logic here.
> > But I am fine to add it if you think it would be better.
> >
>
> I think this is tricky because it is possible that all active workers
> are busy with long-running transactions, so, I think stopping them
> doesn't make sense.

Right, we should not stop running parallel workers.

> I think as long as we are freeing them after use
> it seems okay to me. OTOH, each time after finishing the transaction,
> we can stop the workers, if the workers in the free pool exceed
> 'max_parallel_apply_workers_per_subscription'.

Or the apply leader worker can check that after reloading the config file.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com