Re: Sorting writes during checkpoint

On Tue, 15 Apr 2008, ITAGAKI Takahiro wrote:

> 2x Quad core Xeon, 16GB RAM, 4x HDD (RAID-0)

What is the disk controller in this system? I'm specifically curious
about what write cache was involved, so I can get a better feel for the
hardware your results came from.

I'm busy rebuilding my performance testing systems right now, once that's
done I can review this on a few platforms. One thing that jumped out at
me just reading the code is this happening inside BufferSync:

buf_to_write = (BufAndTag *) palloc(NBuffers * sizeof(BufAndTag));

If shared_buffers(=NBuffers) is set to something big, this could give some
memory churn. And I think it's a bad idea to allocate something this
large at checkpoint time, because what happens if that fails? Really not
the time you want to discover there's no RAM left.

Since you're always going to need this much memory for the system to
operate, and the current model has the system running a checkpoint >50% of
the time, the only thing that makes sense to me is to allocate it at
server start time once and be done with it. That should improve
performance over the original patch as well.

BufAndTag is a relatively small structure (5 ints). Let's call it 40
bytes; even that's only a 0.5% overhead relative to the shared buffer
allocation. If we can speed checkpoints significantly with that much
overhead it sounds like a good tradeoff to me.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD