Re: Logical replication keepalive flood

At Thu, 10 Jun 2021 15:00:16 +0900 (JST), Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com> wrote in
> At Wed, 9 Jun 2021 17:32:25 +0500, Abbas Butt <abbas(dot)butt(at)enterprisedb(dot)com> wrote in
> >
> > On Wed, Jun 9, 2021 at 2:30 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> > > Is it possible that the write/flush location is not
> > > updated at the pace at which we expect?
>
> Yes. MyWalSnd->flush/write are updated far frequently but still
> MyWalSnd->write is behind sentPtr by from thousands of bytes up to
> less than 1 block (1block = 8192 bytes). (Flush lags are larger than
> write lags, of course.)

For more clarity, I changed the previous patch a bit and retook numbers.

Total records: 19476
8: 2 / 4 / 2: 4648 / 302472
16: 5 / 10 / 5: 5427 / 139872
24: 3006 / 6015 / 3028: 4739 / 267215
187: 2 / 0 / 50: 1 / 398

While a 10 seconds run of pgbench, it walsender reads 19476 records
and calls logical_read_xlog_page() 3028 times, and the mean of write
lag is 4739 bytes and flush lag is 267215 bytes (really?), as the
result most of the record fetch causes a keep alive. (The WAL contains
many FPIs).

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center