Re: Improvement of checkpoint IO scheduler for stable transaction responses

(2013/06/17 5:48), Andres Freund wrote:> On 2013-06-16 17:27:56 +0300, Heikki
Linnakangas wrote:
>> If we don't mind scanning the buffer cache several times, we don't
>> necessarily even need to sort the writes for that. Just scan the buffer
>> cache for all buffers belonging to relation A, then fsync it. Then scan the
>> buffer cache again, for all buffers belonging to relation B, then fsync
>> that, and so forth.
>
> That would end up with quite a lot of scans in a reasonably sized
> machines. Not to talk of those that have a million+ relations. That
> doesn't seem to be a good idea for bigger shared_buffers. C.f. the stuff
> we did for 9.3 to make it cheaper to drop a bunch of relations at once
> by only scanning shared_buffers once.
As I written to reply to Heikki, I think that it is unnecessary to exactly buffer
sort which has expensive cost. What we need to solve this problem, we need
accuracy of sort which can be optimized in OS IO scheduler. And we normally have
two optimized IO scheduler layer which are OS layer and RAID controller layer. I
think that performance will be improved if it enables sort accuracy to optimize
in these process. I think that computational complexity required to solve this
problem is one sequential buffer descriptor scan for roughly buffer sort. I will
try to study about this implementation, too.

Best regards,
--
Mitsumasa KONDO
NTT Open Source Software Center