Re: patch for new feature: Buffer Cache Hibernation

On 05/05/2011 05:06 AM, Mitsuru IWASAKI wrote:
> In summary, PgFincore's target is File System Buffer Cache, Buffer
> Cache Hibernation's target is DB Buffer Cache(shared buffers).
>

Right. The thing to realize is that shared_buffers is becoming a
smaller fraction of the total RAM used by the database every year. On
Windows it's been stuck at useful settings being less than 512MB for a
while now. And on UNIX systems, around 8GB seems to be effective upper
limit. Best case, shared_buffers is only going to be around 25% of
total RAM; worst-case, approximately, you might have Windows server with
64GB of RAM where shared_buffers is less than 1% of total RAM.

There's nothing wrong with the general idea you're suggesting. It's
just only targeting a small (and shrinking) subset of the real problem
here. Rebuilding cache state starts with shared_buffers, but that's not
enough of the problem to be an effective tweak on many systems.

I think that all the complexity with CRCs etc. is unlikely to lead
anywhere too, and those two issues are not completely unrelated. The
simplest, safest thing here is the right way to approach this, not the
most complicated one, and a simpler format might add some flexibility
here to reload more cache state too. The bottleneck on reloading the
cache state is reading everything from disk. Trying to micro-optimize
any other part of that is moving in the wrong direction to me. I doubt
you'll ever measure a useful benefit that overcomes the expense of
maintaining the code. And you seem to be moving to where someone can't
restore cache state when they change shared_buffers. A simpler
implementation might still work in that situation; reload until you run
out of buffers if shared_buffers shrinks, reload until you're done with
the original size.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us