Re: Perform streaming logical transactions by background workers and parallel apply

From: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
To: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>
Cc: "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>, "wangw(dot)fnst(at)fujitsu(dot)com" <wangw(dot)fnst(at)fujitsu(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, "shiy(dot)fnst(at)fujitsu(dot)com" <shiy(dot)fnst(at)fujitsu(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Perform streaming logical transactions by background workers and parallel apply
Date: 2022-12-07 04:28:54
Message-ID: CAA4eK1JrwZDqEoc75uQqT=5OjAfHZGbaOBwB12DsAsPPpsF=VQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Dec 7, 2022 at 9:00 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>
> On Thu, Dec 1, 2022 at 7:17 PM houzj(dot)fnst(at)fujitsu(dot)com
> <houzj(dot)fnst(at)fujitsu(dot)com> wrote:
> >
> > > ---
> > > if (am_parallel_apply_worker() && on_subinfo_change) {
> > > /*
> > > * If a parallel apply worker exits due to the subscription
> > > * information change, we notify the leader apply worker so that the
> > > * leader can report more meaningful message in time and restart the
> > > * logical replication.
> > > */
> > > pq_putmessage('X', NULL, 0);
> > > }
> > >
> > > and
> > >
> > > ereport(ERROR,
> > > (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
> > > errmsg("logical replication parallel apply worker exited
> > > because of subscription information change")));
> > >
> > > Do we really need an additional message in case of 'X'? When we call
> > > apply_worker_clean_exit with on_subinfo_change = true, we have reported the
> > > error message such as:
> > >
> > > ereport(LOG,
> > > (errmsg("logical replication parallel apply worker for subscription
> > > \"%s\" will stop because of a parameter change",
> > > MySubscription->name)));
> > >
> > > I think that reporting a similar message from the leader might not be
> > > meaningful for users.
> >
> > The intention is to let leader report more meaningful message if a worker
> > exited due to subinfo change. Otherwise, the leader is likely to report an
> > error like " lost connection ... to parallel apply worker" when trying to send
> > data via shared memory if the worker exited. What do you think ?
>
> Agreed. But do we need to have the leader exit with an error in spite
> of the fact that the worker cleanly exits? If the leader exits with an
> error, the subscription will be disabled if disable_on_error is true,
> right?
>

Right, but the leader will anyway exit at some point either due to an
ERROR like "lost connection ... to parallel worker" or with a LOG
like: "... will restart because of a parameter change" but I see your
point. So, will it be better if we have a LOG message here and then
proc_exit()? Do you have something else in mind for this?

--
With Regards,
Amit Kapila.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Masahiko Sawada 2022-12-07 04:39:24 Re: Perform streaming logical transactions by background workers and parallel apply
Previous Message shiy.fnst@fujitsu.com 2022-12-07 04:05:03 RE: Avoid streaming the transaction which are skipped (in corner cases)