Re: shared-memory based stats collector

At Tue, 08 Sep 2020 17:55:57 +0900 (JST), Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com> wrote in
> At Tue, 08 Sep 2020 17:01:47 +0900 (JST), Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com> wrote in
> > At Thu, 3 Sep 2020 13:16:59 -0400, Stephen Frost <sfrost(at)snowman(dot)net> wrote in> > I've looked through (some of) this thread and through the patches also
> > > and hope to provide a review of the bits that should be targetting v14
> > > (unlike the above) soon.
> >
> > Thanks. Current the patch is found to lead to write contention heavier
> > than the current stats collector when nearly a thousand backend
> > heavily write to the same table. I need to address that.
> >
> > - Collect stats via sockets (in the same way as the current implement)
> > and write/read to/from shared memory.
> >
> > - Use our own lock-free (maybe) ring-buffer before stats-write enters
> > lock-waiting mode, then new stats collector(!) process consumes the
> > queue.
> >
> > - Some other measures..

- Make dshash search less frequent. To find the actual source of the
contension, We're going to measure performance with the attached on
top of the latest patch let sessions cache the result of dshash
searches for the session lifetime. (table-dropping vacuum clears the
local hash.)

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center