Re: Failed transaction statistics to measure the logical replication progress

On Tue, Dec 14, 2021 at 11:28 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Mon, Dec 13, 2021 at 5:48 PM osumi(dot)takamichi(at)fujitsu(dot)com
> <osumi(dot)takamichi(at)fujitsu(dot)com> wrote:
> >
> > On Monday, December 13, 2021 6:19 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> > > On Tue, Dec 7, 2021 at 3:12 PM osumi(dot)takamichi(at)fujitsu(dot)com
> > > <osumi(dot)takamichi(at)fujitsu(dot)com> wrote:
> > >
> > > Few questions and comments:
> > Thank you for your comments !
> >
> > > ========================
> > > 1.
> > > The <structname>pg_stat_subscription_workers</structname> view will
> > > contain
> > > one row per subscription worker on which errors have occurred, for workers
> > > applying logical replication changes and workers handling the initial data
> > > - copy of the subscribed tables. The statistics entry is removed when the
> > > - corresponding subscription is dropped.
> > > + copy of the subscribed tables. Also, the row corresponding to the apply
> > > + worker shows all transaction statistics of both types of workers on the
> > > + subscription. The statistics entry is removed when the corresponding
> > > + subscription is dropped.
> > >
> > > Why did you choose to show stats for both types of workers in one row?
> > This is because if we have hundreds or thousands of tables for table sync,
> > we need to create many entries to cover them and store the entries for all tables.
> >
>
> If we fear a large number of entries for such workers then won't it be
> better to show the value of these stats only for apply workers. I
> think normally the table sync workers perform only copy operation or
> maybe a fixed number of xacts, so, one might not be interested in the
> transaction stats of these workers. I find merging only specific stats
> of two different types of workers confusing.
>
> What do others think about this?

I understand the concern to have a large number of entries but I agree
that merging only specific stats would confuse users. As Amit
suggested, it'd be better to show only apply workers' transaction
stats.

Regards,

--
Masahiko Sawada
EDB: https://www.enterprisedb.com/