Re: Perform streaming logical transactions by background workers and parallel apply

From: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>, "wangw(dot)fnst(at)fujitsu(dot)com" <wangw(dot)fnst(at)fujitsu(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, "shiy(dot)fnst(at)fujitsu(dot)com" <shiy(dot)fnst(at)fujitsu(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Perform streaming logical transactions by background workers and parallel apply
Date: 2022-12-07 12:48:39
Message-ID: CAD21AoAzYstJVM0nMVnXZoeYamqD2j92DkWVH=YbGtA4yzy19A@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Dec 7, 2022 at 4:31 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Wed, Dec 7, 2022 at 10:10 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> >
> > On Wed, Dec 7, 2022 at 1:29 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> > >
> > > Right, but the leader will anyway exit at some point either due to an
> > > ERROR like "lost connection ... to parallel worker" or with a LOG
> > > like: "... will restart because of a parameter change" but I see your
> > > point. So, will it be better if we have a LOG message here and then
> > > proc_exit()? Do you have something else in mind for this?
> >
> > No, I was thinking that too. It's better to write a LOG message and do
> > proc_exit().
> >
> > Regarding the error "lost connection ... to parallel worker", it could
> > still happen depending on the timing even if the parallel worker
> > cleanly exits due to parameter changes, right? If so, I'm concerned
> > that it could lead to disable the subscription unexpectedly if
> > disable_on_error is enabled.
> >
>
> If we want to avoid this then I think we have the following options
> (a) parallel apply skips checking parameter change (b) parallel worker
> won't exit on parameter change but will silently absorb the parameter
> and continue its processing; anyway, the leader will detect it and
> stop the worker for the parameter change
>
> Among these, the advantage of (b) is that it will allow reflecting the
> parameter change (that doesn't need restart) in the parallel worker.
> Do you have any better idea to deal with this?

I think (b) is better. We need to reflect the synchronous_commit
parameter also in parallel workers in the worker pool.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message houzj.fnst@fujitsu.com 2022-12-07 13:03:37 RE: Perform streaming logical transactions by background workers and parallel apply
Previous Message Alvaro Herrera 2022-12-07 12:20:41 Re: Support logical replication of DDLs