Re: Clock sweep not caching enough B-Tree leaf pages?

On 2014-04-17 21:44:47 +0300, Heikki Linnakangas wrote:
> On 04/17/2014 09:38 PM, Stephen Frost wrote:
> >* Greg Stark (stark(at)mit(dot)edu) wrote:
> >>On Thu, Apr 17, 2014 at 12:21 PM, Stephen Frost <sfrost(at)snowman(dot)net> wrote:
> >>>Ehhh. No. If it's a hot page that we've been holding in *our* cache
> >>>long enough, the kernel will happily evict it as 'cold' from *its*
> >>>cache, leading to...
> >>
> >>This is a whole nother problem.
> >>
> >>It is worrisome that we could be benchmarking the page replacement
> >>algorithm in Postgres and choose a page replacement algorithm that
> >>chooses pages that performs well because it tends to evict pages that
> >>are in the OS cache. And then one day (hopefully not too far off)
> >>we'll fix the double buffering problem and end up with a strange
> >>choice of page replacement algorithm.
> >
> >That's certainly possible but I don't see the double buffering problem
> >going away any time particularly soon and, even if it does, it's likely
> >to either a) mean we're just using the kernel's cache (eg: something w/
> >mmap, etc), or b) will involve so many other changes that this will end
> >up getting changed anyway. In any case, while I think we should
> >document any such cache management system we employ as having this risk,
> >I don't think we should worry about it terribly much.
>
> Note that if we somehow come up with a page replacement algorithm that tends
> to evict pages that are in the OS cache, we have effectively solved the
> double buffering problem. When a page is cached in both caches, evicting it
> from one of them eliminates the double buffering. Granted, you might prefer
> to evict it from the OS cache instead, and such an algorithm could be bad in
> other ways. But if a page replacement algorithm happens avoid double
> buffering, that's a genuine merit for that algorithm.

I don't think it's a good idea to try to synchronize algorithms with the
OSs. There's so much change about the caching logic in e.g. linux that
it won't stay effective for very long.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services