Remove wal_[sync|write][_time] from pg_stat_wal and track_wal_io_timing

From: Bertrand Drouvot <bertranddrouvot(dot)pg(at)gmail(dot)com>
To: pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Remove wal_[sync|write][_time] from pg_stat_wal and track_wal_io_timing
Date: 2025-02-18 10:43:15
Message-ID: Z7RkQ0EfYaqqjgz/@ip-10-97-1-34.eu-west-3.compute.internal
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi hackers,

a051e71e28a added the WAL writes and fsyncs information (and their related timing)
into the pg_stat_io view so that one can now find the same information in 2 places:
pg_stat_wal and pg_stat_io (for the wal object). This lead to a discussion in
[1] to remove this information from the pg_stat_wal view.

=== Let's sump up in this dedicated thread what has been discussed in [1]

1. The pg_stat_io's writes and fsyncs (and their relative timing) are incremented
at the exact same places as their pg_stat_wal counterparts.

2. pgstat_tracks_io_bktype() returns false for backend's that do not generate WAL
(so that we don't lost WAL information in pg_stat_io).

3. pg_stat_stat.wal_write is the same value as "select sum(writes)
from pg_stat_io where object = 'wal' and context = 'normal'" as these
are incremented in XLogWrite().

4. Same argument about pg_stat_wal.wal_write_time with
pg_stat_io.write_time.

5. issue_xlog_fsync() tells that pg_stat_wal.wal_sync_time and
sum(pg_stat_io.fsync_time) under object=wal and context=normal are the
same values.

6. Same argument with the fsync counters pg_stat_wal.wal_sync and
pg_stat_io.fsyncs.

7. This will encourage monitoring pull to move to pg_stat_io, where there is much
more context and granularity of the stats data.

8. That will simplify the work done for the per backend WAL statistics ([1])

=== Remarks

R1. The "bytes" fields differ (as pg_stat_wal.wal_bytes somehow "focus" on the wal
records size while the pg_stat_io's unit is the wal_block_size) so we keep them
in both places.

R2. track_wal_io_timing becomes useless once those fields are gone from pg_stat_wal

R3. PendingWalStats becomes empty, so removes it.

=== Patch structure

PFA 3 sub patches:

- 0001: Add details in the pg_stat_io doc about the wal object. That's basically
more or less a copy/paste from the pg_stat_wal fields description that will be
removed in 0002. As we already track them in pg_stat_io, it's done in a dedicated
patch.

- 0002: Remove wal_[sync|write][_time] from pg_stat_wal, PendingWalStats and
track_wal_io_timing. That does not make sense to split this work in sub-patches
so that all of this in done in 0002.

- 0003: remove the pgstat_prepare_io_time() parameter. Now that track_wal_io_timing
is gone there is no need for pgstat_prepare_io_time() to get a parameter.

[1]: https://www.postgresql.org/message-id/Z7NSc5j6M4g2r1HD%40ip-10-97-1-34.eu-west-3.compute.internal

Looking forward to your feedback,

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachment Content-Type Size
v1-0001-Add-details-in-the-pg_stat_io-doc-about-the-wal-o.patch text/x-diff 2.9 KB
v1-0002-Remove-wal_-sync-write-_time-from-pg_stat_wal-and.patch text/x-diff 20.1 KB
v1-0003-remove-the-pgstat_prepare_io_time-parameter.patch text/x-diff 8.3 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Bertrand Drouvot 2025-02-18 10:45:29 Re: per backend WAL statistics
Previous Message Shlok Kyal 2025-02-18 10:29:43 Re: Restrict publishing of partitioned table with a foreign table as partition