| From: | Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com> |
|---|---|
| To: | sfrost(at)snowman(dot)net |
| Cc: | alvherre(at)2ndquadrant(dot)com, andres(at)anarazel(dot)de, michael(at)paquier(dot)xyz, thomas(dot)munro(at)gmail(dot)com, tomas(dot)vondra(at)2ndquadrant(dot)com, ah(at)cybertec(dot)at, magnus(at)hagander(dot)net, robertmhaas(at)gmail(dot)com, tgl(at)sss(dot)pgh(dot)pa(dot)us, pgsql-hackers(at)postgresql(dot)org |
| Subject: | Re: shared-memory based stats collector |
| Date: | 2020-09-08 08:55:57 |
| Message-ID: | 20200908.175557.617150409868541587.horikyota.ntt@gmail.com |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
At Tue, 08 Sep 2020 17:01:47 +0900 (JST), Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com> wrote in
> At Thu, 3 Sep 2020 13:16:59 -0400, Stephen Frost <sfrost(at)snowman(dot)net> wrote in> > I've looked through (some of) this thread and through the patches also
> > and hope to provide a review of the bits that should be targetting v14
> > (unlike the above) soon.
>
> Thanks. Current the patch is found to lead to write contention heavier
> than the current stats collector when nearly a thousand backend
> heavily write to the same table. I need to address that.
>
> - Collect stats via sockets (in the same way as the current implement)
> and write/read to/from shared memory.
>
> - Use our own lock-free (maybe) ring-buffer before stats-write enters
> lock-waiting mode, then new stats collector(!) process consumes the
> queue.
>
> - Some other measures..
>
> Anyway, I'll post a rebased version soon.
This is that. I'll continue seeking a way to solve the contention.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
| Attachment | Content-Type | Size |
|---|---|---|
| v36-0001-Use-standard-crash-handler-in-archiver.patch | text/x-patch | 1.7 KB |
| v36-0002-sequential-scan-for-dshash.patch | text/x-patch | 8.4 KB |
| v36-0003-Add-conditional-lock-feature-to-dshash.patch | text/x-patch | 6.2 KB |
| v36-0004-Make-archiver-process-an-auxiliary-process.patch | text/x-patch | 17.6 KB |
| v36-0005-Shared-memory-based-stats-collector.patch | text/x-patch | 257.6 KB |
| v36-0006-Doc-part-of-shared-memory-based-stats-collector.patch | text/x-patch | 20.7 KB |
| v36-0007-Remove-the-GUC-stats_temp_directory.patch | text/x-patch | 9.6 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Konstantin Knizhnik | 2020-09-08 09:06:41 | Re: INSERT ON CONFLICT and RETURNING |
| Previous Message | Michael Paquier | 2020-09-08 08:23:23 | Re: Internal key management system |