Re: per backend I/O statistics

On Fri, Jan 17, 2025 at 10:23:48AM +0000, Bertrand Drouvot wrote:
> Please find attached a patch implementing the ideas above, meaning:
>
> - It creates a new PendingBackendStats variable
> - It uses this variable to increment and flush per backend pending IO statistics
>
> That way we get rid of the memory allocation for pending IO statistics.
>
> One remark: a special case has been added in pgstat_flush_pending_entries(). The
> reason is that while the per backend stats are "variable-numbered" stats, it could
> be that their pending stats are not part of pgStatPending. This is currently the
> case (with this patch) as the per backend pending IO stats are not tracked with
> pgstat_prep_backend_pending() and friends anymore.

+ /*
+ * There is a special case for some pending stats that are tracked in
+ * PendingBackendStats. It's possible that those have not been flushed
+ * above, hence the extra check here.
+ */
+ if (!pg_memory_is_all_zeros(&PendingBackendStats,
+ sizeof(struct PgStat_BackendPending)))
+ {
+ PgStat_EntryRef *entry_ref;
+
+ entry_ref = pgstat_get_entry_ref(PGSTAT_KIND_BACKEND, InvalidOid,
+ MyProcNumber, false, NULL);

Hmm. Such special complexities in pgstat.c are annoying. There is
a stupid thing I am wondering here. For the WAL stats, why couldn't
we place some calls of pgstat_prep_backend_pending() in strategic
places like XLogBeginInsert() to force all the allocation steps of the
pending entry to happen before we would enter the critical sections
when doing a WAL insertion? As far as I can see, there is a special
case with 2PC where XLogBeginInsert() could be called in a critical
section, but that seems to be the only one at quick glance.
--
Michael