Re: patch for new feature: Buffer Cache Hibernation

On Fri, May 6, 2011 at 5:31 PM, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
> I think that all the complexity with CRCs etc. is unlikely to lead anywhere
> too, and those two issues are not completely unrelated.  The simplest,
> safest thing here is the right way to approach this, not the most
> complicated one, and a simpler format might add some flexibility here to
> reload more cache state too.  The bottleneck on reloading the cache state is
> reading everything from disk.  Trying to micro-optimize any other part of
> that is moving in the wrong direction to me.  I doubt you'll ever measure a
> useful benefit that overcomes the expense of maintaining the code.  And you
> seem to be moving to where someone can't restore cache state when they
> change shared_buffers.  A simpler implementation might still work in that
> situation; reload until you run out of buffers if shared_buffers shrinks,
> reload until you're done with the original size.

I don't think there's any need for this to get data into
shared_buffers at all. Getting it into the OS cache oughta be plenty
sufficient, no?

ISTM that a very simple approach here would be to save the contents of
each shared buffer on clean shutdown, and to POSIX_FADV_WILLNEED those
buffers on startup. We could worry about additional complexity, like
using fincore to probe the OS cache, in a follow-on patch. While
reloading only 8GB of maybe 30GB of cached data on restart would not
be as good as reloading all of it, it would be a lot better than
reloading none of it, and the gymnastics required seems substantially
less.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company