Re: Clock sweep not caching enough B-Tree leaf pages?

On 2014-04-17 10:48:15 -0400, Bruce Momjian wrote:
> On Thu, Apr 17, 2014 at 10:40:40AM -0400, Robert Haas wrote:
> > That can happen, but the real problem I was trying to get at is that
> > when all the buffers get up to max usage count, they all appear
> > equally important. But in reality they're not. So when we do start
> > evicting those long-resident buffers, it's essentially random which
> > one we kick out.
>
> True. Ideally we would have some way to know that _all_ the buffers had
> reached the maximum and kick off a sweep to decrement them all. I am
> unclear how we would do that. One odd idea would be to have a global
> counter that is incremented everytime a buffer goes from 4 to 5 (max)
> --- when the counter equals 50% of all buffers, do a clock sweep. Of
> course, then the counter becomes a bottleneck.

I have my doubts that we'll make the current scheme, where buffer
reclaim essentially is O(NBuffers), work much better. Especially as CPU
cache effects make such large, high frequency, accesses really
expensive.
I think we need more drastic changes.

I am *not* suggesting that we do that, but I believe it'd be possible to
implement a full LRU and be faster than today in scenarios with
nontrivial amounts of shared buffers.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services