Re: patch for new feature: Buffer Cache Hibernation

Hi,
Sorry, I missed these messages because I didn't subscribe to this list.
# I've just subscribed temporary

> > I think that all the complexity with CRCs etc. is unlikely to lead anywhere
> > too, and those two issues are not completely unrelated.  The simplest,
> > safest thing here is the right way to approach this, not the most
> > complicated one, and a simpler format might add some flexibility here to
> > reload more cache state too.  The bottleneck on reloading the cache state is
> > reading everything from disk.  Trying to micro-optimize any other part of
> > that is moving in the wrong direction to me.  I doubt you'll ever measure a
> > useful benefit that overcomes the expense of maintaining the code.  And you
> > seem to be moving to where someone can't restore cache state when they
> > change shared_buffers.  A simpler implementation might still work in that
> > situation; reload until you run out of buffers if shared_buffers shrinks,
> > reload until you're done with the original size.
>
> Yeah, I'm pretty well convinced this whole approach is a dead end.
> Priming the OS buffer cache seems way more useful. I also think
> saving the blocks to be read rather than the actual blocks makes a lot
> more sense.

OK, there are two your suggestions here IIUC.
# if not, please correct me.
1. restore buffer blocks based on buffer descriptors, not from the saved file.
2. support restoring cache state even if shared_buffers had changed.

For 1, I've just finish my work. The latest patch is available at:
http://people.freebsd.org/~iwasaki/postgres/buffer-cache-hibernation-postgresql-20110507.patch

On my box, shared_buffers can be set up to only 200MB.
Elapsed time for starting up is almost the same, about 3 sec (w/o
hibernation takes about 1 sec).
For shutdown, writing buffer blocks takes about 10 sec, otherwise
about 1 sec.

Well, it seems you were right :)
By restoring buffer blocks based on buffer descriptors, the OS buffer
cache will be filled too. This can help buffer updating performance
I believe.

I think saving buffer blocks is still useful for debugging or portability,
so I would like to remain the support code in my patch.

For 2, I'm not sure how to implement this.
The problem is that freelist.c:StrategyControl is also restored at
startup, but I have no idea currently how to adjust StrategyControl
when shared_buffer had changed.
StrategyControl has important data on buffer allocation, so this should be
matched with shared_buffer, I belive.

Changing shared_buffer is not so often on production environment.
Current implementation like this;
If shared_buffer had changed, restoring is aborted only on that time
and saving is executed with new shared_buffer at shutdown, restoring
is executed at startup on next time.

I have one more day for working on this, but I may give up...

Thanks