Re: less expensive pg_buffercache on big shmem

On Wed, Aug 31, 2016 at 8:27 PM, Ivan Kartyshov
<i(dot)kartyshov(at)postgrespro(dot)ru> wrote:
> Recently I have finished my work on a patch for pg_buffercache contrib, I
> think it's time to share my results.

Thanks for sharing your results.

> V1.2 implementation contains flexible loop which can collect shared memory
> statistic using three different methods:
> 1) with holding LWLock only on one partition of shared memory
> (semiconsistent method)
> 2) without LWLocks (nonconsistent method),
> 3) or in vanilia way (consistent method)
>
> The aforementioned allow us to launch pg_buffercache in the three different
> ways.

I am a little skeptical about the idea of offering the user three
different choices about how to do this. That seems like it is
exposing what ought to be an internal implementation detail and I'm
not sure users will really know how to make an intelligent choice
between the three options. But of course others may have a different
view on that.

I wonder whether we ought to just switch from the consistent method to
the semiconsistent method and call it good. I agree with you that
taking every buffer partition lock simultaneously seems like too much
locking. And in the future if we replace the buffer mapping hash with
something that is lock-free or close to it, then we wouldn't even have
buffer partition locks any more, and probably no way at all to get an
entirely consistent snapshot.

What do you think of this?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company