Re: Replication slot stats misgivings

On Wed, Apr 28, 2021 at 5:01 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Wed, Apr 28, 2021 at 4:51 PM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> >
> > On Wed, Apr 28, 2021 at 6:39 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> @@ -1369,7 +1369,7 @@ ReorderBufferIterTXNNext(ReorderBuffer *rb,
> ReorderBufferIterTXNState *state)
> * Update the total bytes processed before releasing the current set
> * of changes and restoring the new set of changes.
> */
> - rb->totalBytes += rb->size;
> + rb->totalBytes += entry->txn->total_size;
> if (ReorderBufferRestoreChanges(rb, entry->txn, &entry->file,
> &state->entries[off].segno))
>
> I have not tested this but won't in the above change you need to check
> txn->toptxn for subtxns?
>

Now, I am able to reproduce this issue:
Create table t1(c1 int);
select pg_create_logical_replication_slot('s', 'test_decoding');
Begin;
insert into t1 values(1);
savepoint s1;
insert into t1 select generate_series(1, 100000);
commit;

postgres=# select count(*) from pg_logical_slot_peek_changes('s1', NULL, NULL);
count
--------
100005
(1 row)

postgres=# select * from pg_stat_replication_slots;
slot_name | spill_txns | spill_count | spill_bytes | stream_txns |
stream_count | stream_bytes | total_txns | total_bytes |
stats_reset
-----------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+----------------------------------
s1 | 0 | 0 | 0 | 0 |
0 | 0 | 2 | 13200672 | 2021-04-29
14:33:55.156566+05:30
(1 row)

select * from pg_stat_reset_replication_slot('s1');

Now reduce the logical decoding work mem to allow spilling.
postgres=# set logical_decoding_work_mem='64kB';
SET
postgres=# select count(*) from pg_logical_slot_peek_changes('s1', NULL, NULL);
count
--------
100005
(1 row)

postgres=# select * from pg_stat_replication_slots;
slot_name | spill_txns | spill_count | spill_bytes | stream_txns |
stream_count | stream_bytes | total_txns | total_bytes |
stats_reset
-----------+------------+-------------+-------------+-------------+--------------+--------------+------------+-------------+----------------------------------
s1 | 1 | 202 | 13200000 | 0 |
0 | 0 | 2 | 672 | 2021-04-29
14:35:21.836613+05:30
(1 row)

You can notice that after we have allowed spilling the 'total_bytes'
stats is showing a different value. The attached patch fixes the issue
for me. Let me know what do you think about this?

--
With Regards,
Amit Kapila.