Re: SLRUs in the main buffer pool - Page Header definitions

On Fri, Sep 8, 2023 at 8:56 AM Stephen Frost <sfrost(at)snowman(dot)net> wrote:
> If we're going to effectively segregate the buffer pool into SLRU parts
> vs. everything else and then use the existing strategies for SLRUs and
> have that be different from what everything else is using ... then
> that doesn't seem like it's really changing things. What would be the
> point of moving the SLRUs into the main buffer pool then?

I think that integrating the SLRUs into the same memory allocation as
shared_buffers could potentially be pretty helpful even if the buffers
are managed differently. For instance, each SLRU could evaluate (using
some algorithm) whether it needs more or fewer buffers than it has
currently and either steal more buffers from shared_buffers or give
some back. That does seem less elegant than having everything running
through a single system, but getting to the point where everything
runs through a single system may be difficult, and such an
intermediate state could have a lot of advantages with, perhaps, less
risk of breaking stuff.

As far as I am aware, the optimal amount of memory for any particular
SLRU is almost always quite small compared to shared_buffers, but it
can be quite large compared to what we allocate currently. To do
anything about that, we clearly need to fix the algorithms to scale
better. But even once we've done that, I don't think we want to
allocate the largest amount of clog buffers that anyone could ever
need in all instances, and the same for every other SLRU. That seems
wasteful. We could expose a separate tunable for each one, but that is
a lot for users to tune correctly, and it implies using the same value
for the whole lifetime of the server. Letting the value float around
dynamically would make a lot of sense especially for things like
pg_multixact -- if there's a lot of multixacts, grab some more
buffers, if there aren't, release some or even all of them. The same
kind of thing can happen with other SLRUs -- e.g. as the oldest
running xact gets older, you need more subxact buffers; when
long-running transactions end, you need fewer.

Again, I'm not sure what the right thing to do here actually is, just
that I wouldn't be too quick to reject a partial integration into
shared_buffers.

--
Robert Haas
EDB: http://www.enterprisedb.com