Re: Design of pg_stat_subscription_workers vs pgstats

From: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Peter Eisentraut <peter(dot)eisentraut(at)enterprisedb(dot)com>, "David G(dot) Johnston" <david(dot)g(dot)johnston(at)gmail(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Design of pg_stat_subscription_workers vs pgstats
Date: 2022-02-16 08:45:03
Message-ID: CAA4eK1KuaYmyg_fHi8xteAXuNBQTGkmAy7FG6xMwxZfEuoNFBw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Feb 15, 2022 at 11:47 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
>
> Hi,
>
> On 2022-02-04 09:23:06 +0530, Amit Kapila wrote:
> > On Thu, Feb 3, 2022 at 3:25 PM Peter Eisentraut
> > <peter(dot)eisentraut(at)enterprisedb(dot)com> wrote:
> > >
> > > On 02.02.22 07:54, Amit Kapila wrote:
> > >
> > > > Sure, but is this the reason you want to store all the error info in
> > > > the system catalog? I agree that providing more error info could be
> > > > useful and also possibly the previously failed (apply) xacts info as
> > > > well but I am not able to see why you want to have that sort of info
> > > > in the catalog. I could see storing info like err_lsn/err_xid that can
> > > > allow to proceed to apply worker automatically or to slow down the
> > > > launch of errored apply worker but not all sort of other error info
> > > > (like err_cnt, err_code, err_message, err_time, etc.). I want to know
> > > > why you are insisting to make all the error info persistent via the
> > > > system catalog?
> > >
> > > Let's flip this around and ask, why not?
> > >
> >
> > Because we don't necessarily need all this information after the crash
> > and neither is this information about any system object which we
> > require for performing operations on objects.
>
> I find this not particularly convincing. IMO data that leads the user to
> compromise "replication integrity" is pretty crucial.
>
> And skipped data needs to be logged somewhere persistent, so that there's a
> chance to analyze / recover.
>

Valid point. I think we can store this in a separate table
(pg_subsciption_conflict_history or something like that) but at some
point we need to clear this data like say when the user drops the
subscription or maybe a separate interface altogether or after a
particular time interval (user-configurable or otherwise), the
subscription worker or some other background worker clears this
information.

--
With Regards,
Amit Kapila.

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2022-02-16 08:49:37 Re: Design of pg_stat_subscription_workers vs pgstats
Previous Message John Naylor 2022-02-16 08:43:12 Re: do only critical work during single-user vacuum?