Re: shared-memory based stats collector

From: Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>
To: masao(dot)fujii(at)oss(dot)nttdata(dot)com
Cc: andres(at)anarazel(dot)de, gkokolatos(at)protonmail(dot)com, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: shared-memory based stats collector
Date: 2021-03-18 07:56:02
Message-ID: 20210318.165602.1641754489159242006.horikyota.ntt@gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

At Tue, 16 Mar 2021 10:27:55 +0900 (JST), Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com> wrote in
> At Mon, 15 Mar 2021 17:49:36 +0900 (JST), Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com> wrote in
> > Thanks for committing this! I'm very happy to see this reduces the
> > size of this patchset.
>
> Now that 0003 is committed as d75288fb27, and 33394ee6f2 conflicts
> with old 0004, I'd like to post a rebased version for future work.
>
> The commit 33394ee6f2 adds on-exit forced write of WAL stats on
> walwriter and in this patch that part would appear to have been
> removed. However, this patchset already does that by calling to
> pgstat_report_stat from pgstat_beshutdown_hook.

Rebased and fixed two bugs. Not addressed received comments in this
version.

5f79580ad6 confilicts with this. However, the modified code is removed
by the patch. (And 081876d75e made a small conflict.)

I fixed two silly bugs along with the rebasing.

1) An assertion failure happens while accessing pg_stat_database,
since pgstat_fetch_stat_dbentry() rejected 0 as database oid.

2) pgstat_report_stat() failed to flush out global database stats (oid
= 0). I stopped to collecgt database entries in the existing loop on
pgStatLocalhash, then modified the existing second loop to scan
pgStatLocalhash directly for database stats entries. The second loop
might get slow when the first loop left many table/function local
stats entreis due to heavy load.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

Attachment Content-Type Size
v57-0001-sequential-scan-for-dshash.patch text/x-patch 9.3 KB
v57-0002-Add-conditional-lock-feature-to-dshash.patch text/x-patch 6.2 KB
v57-0003-Shared-memory-based-stats-collector.patch text/x-patch 312.4 KB
v57-0004-Doc-part-of-shared-memory-based-stats-collector.patch text/x-patch 19.7 KB
v57-0005-Remove-the-GUC-stats_temp_directory.patch text/x-patch 13.7 KB
v57-0006-Exclude-pg_stat-directory-from-base-backup.patch text/x-patch 1.2 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2021-03-18 08:14:24 Re: Proposal: Save user's original authenticated identity for logging
Previous Message Dean Rasheed 2021-03-18 07:54:38 Re: PoC/WIP: Extended statistics on expressions