From: | Melanie Plageman <melanieplageman(at)gmail(dot)com> |
---|---|
To: | Lukas Fittl <lukas(at)fittl(dot)com> |
Cc: | Andres Freund <andres(at)anarazel(dot)de>, Justin Pryzby <pryzby(at)telsasoft(dot)com>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>, Magnus Hagander <magnus(at)hagander(dot)net>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Maciek Sakrejda <m(dot)sakrejda(at)gmail(dot)com> |
Subject: | Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?) |
Date: | 2022-10-06 22:23:53 |
Message-ID: | CAAKRu_Yu7KKKA8YRsQP1wuU36Ypm7BkYu2yDoc67kdR07zG_Gw@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
v31 failed in CI, so
I've attached v32 which has a few issues fixed:
- addressed some compiler warnings I hadn't noticed locally
- autovac launcher and worker do indeed use bulkread strategy if they
end up starting before critical indexes have loaded and end up doing a
sequential scan of some catalog tables, so I have changed the
restrictions on BackendTypes allowed to track IO Operations in
IOCONTEXT_BULKREAD
- changed the name of the column "fsynced" to "files_synced" to make it
more clear what unit it is in (and that the unit differs from that of
the "unit" column)
In an off-list discussion with Andres, he mentioned that he thought
buffers reused by a BufferAccessStrategy should be split from buffers
"acquired" and that "acquired" should be renamed "clocksweeps".
I have started doing this, but for BufferAccessStrategy IO there are a
few choices about how we want to count the clocksweeps:
Currently the following situations are counted under the following
IOContexts and IOOps:
IOCONTEXT_[VACUUM,BULKREAD,BULKWRITE], IOOP_ACQUIRE
- reuse a buffer from the ring
IOCONTEXT_SHARED, IOOP_ACQUIRE
- add a buffer to the strategy ring initially
- add a new shared buffer to the ring when all the existing buffers in
the ring are pinned
And in the new paradigm, I think these are two good options:
1)
IOCONTEXT_[VACUUM,BULKREAD,BULKWRITE], IOOP_CLOCKSWEEP
- add a buffer to the strategy ring initially
- add a new shared buffer to the ring when all the existing buffers in
the ring are pinned
IOCONTEXT_[VACUUM,BULKREAD,BULKWRITE], IOOP_REUSE
- reuse a buffer from the ring
2)
IOCONTEXT_[VACUUM,BULKREAD,BULKWRITE], IOOP_CLOCKSWEEP
- add a buffer to the strategy ring initially
IOCONTEXT_[VACUUM,BULKREAD,BULKWRITE], IOOP_REUSE
- reuse a buffer from the ring
IOCONTEXT SHARED, IOOP_CLOCKSWEEP
- add a new shared buffer to the ring when all the existing buffers in
the ring are pinned
However, if we want to differentiate between buffers initially added to
the ring and buffers taken from shared buffers and added to the ring
because all strategy ring buffers are pinned or have a usage count above
one, then we would need to either do so inside of GetBufferFromRing() or
propagate this distinction out somehow (easy enough if we care to do
it).
There are other combinations that I could come up with a justification
for as well, but I wanted to know what other people thought made sense
(and would make sense to users).
- Melanie
Attachment | Content-Type | Size |
---|---|---|
v32-0002-Aggregate-IO-operation-stats-per-BackendType.patch | text/x-patch | 21.9 KB |
v32-0001-Track-IO-operation-statistics-locally.patch | text/x-patch | 26.9 KB |
v32-0003-Add-system-view-tracking-IO-ops-per-backend-type.patch | text/x-patch | 38.3 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | David Rowley | 2022-10-06 22:38:05 | Re: Reducing the chunk header sizes on all memory context types |
Previous Message | Tom Lane | 2022-10-06 21:57:01 | Re: Reducing the chunk header sizes on all memory context types |