RE: Failed transaction statistics to measure the logical replication progress

From: "wangw(dot)fnst(at)fujitsu(dot)com" <wangw(dot)fnst(at)fujitsu(dot)com>
To: "osumi(dot)takamichi(at)fujitsu(dot)com" <osumi(dot)takamichi(at)fujitsu(dot)com>
Cc: Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, "vignesh21(at)gmail(dot)com" <vignesh21(at)gmail(dot)com>, "amit(dot)kapila16(at)gmail(dot)com" <amit(dot)kapila16(at)gmail(dot)com>, "sawada(dot)mshk(at)gmail(dot)com" <sawada(dot)mshk(at)gmail(dot)com>, "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>, 'Greg Nancarrow' <gregn4422(at)gmail(dot)com>
Subject: RE: Failed transaction statistics to measure the logical replication progress
Date: 2021-12-22 12:37:35
Message-ID: OS3PR01MB6275734F2CCB71ACFE9074509E7D9@OS3PR01MB6275.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Dec 22, 2021 at 6:14 PM osumi(dot)takamichi(at)fujitsu(dot)com <osumi(dot)takamichi(at)fujitsu(dot)com> wrote:
>
> Attached the new patch v19.
>

I have a question on the v19-0002 patch:

When I tested for this patch, I found pg_stat_subscription_workers has some unexpected data.
For example:
[Publisher]
create table replica_test1(a int, b text); create publication pub1 for table replica_test1;
create table replica_test2(a int, b text); create publication pub2 for table replica_test2;

[Subscriber]
create table replica_test1(a int, b text); create subscription sub1 CONNECTION 'dbname=postgres' publication pub1;
create table replica_test2(a int, b text); create subscription sub2 CONNECTION 'dbname=postgres' publication pub2;

[Publisher]
insert into replica_test1 values(1,'1');

[Subscriber]
select * from pg_stat_subscription_workers;

-[ RECORD 1 ]------+------
Subid | 16389
subname | sub1
subrelid |
commit_count | 1
...
-[ RECORD 2 ]------+------
subid | 16395
subname | sub2
subrelid |
commit_count | 1
...

I originally expected only one record for "sub1".

I think the reason is apply_handle_commit() always invoke pgstat_report_subworker_xact_end().
But when we insert data to replica_test1 in publish side:
[In the publish]
pub1's walsender1 will send three messages((LOGICAL_REP_MSG_BEGIN, LOGICAL_REP_MSG_INSERT and LOGICAL_REP_MSG_COMMIT))
to sub1's apply worker1.
pub2's walsender2 will also send two messages(LOGICAL_REP_MSG_BEGIN and LOGICAL_REP_MSG_COMMIT)
to sub2's apply worker2. Because inserted table is not published by pub2.

[In the subscription]
sub1's apply worker1 receive LOGICAL_REP_MSG_COMMIT,
so invoke pgstat_report_subworker_xact_end to increase commit_count of sub1's stats.
sub2's apply worker2 receive LOGICAL_REP_MSG_COMMIT,
it will do the same action to increase commit_count of sub2's stats.

Do we expect these commit counts which come from empty transactions ?

Regards,
Wang wei

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2021-12-22 12:52:01 Re: parallel vacuum comments
Previous Message Tomas Vondra 2021-12-22 12:11:54 Re: sequences vs. synchronous replication