Re: Why we are going to have to go DirectIO

(2013/12/04 11:28), Tatsuo Ishii wrote:
>> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>>> On Tue, Dec 3, 2013 at 11:44 PM, Josh Berkus <josh(at)agliodbs(dot)com> wrote:
>>>> Would certainly be nice. Realistically, getting good automated
>>>> performace tests will require paying someone like Greg S., Mark or me
>>>> for 6 solid months to develop them, since worthwhile open source
>>>> performance test platforms currently don't exist. That money has never
>>>> been available; maybe I should do a kickstarter.
>>
>>> So in order to get *testing* we need to pay somebody. But to build a great
>>> database server, we can rely on volunteer efforts or sponsorship from
>>> companies who are interested in moving the project forward?
>>
>> And even more to the point, volunteers to reinvent the kernel I/O stack
>> can be found on every street corner? And those volunteers won't need any
>> test scaffolding to be sure that *their* version never has performance
>> regressions? (Well, no, they won't, because no such thing will ever be
>> built. But we do need better test scaffolding for real problems.)
>
> Can we avoid the Linux kernel problem by simply increasing our shared
> buffer size, say up to 80% of memory?
It will be swap more easier.

I think that we should use latest system-calls in Linux which are like
posix_fadvise(), fallocate() and sync_file_range() etc, when we use linux
buffered IO. Hoevere, PostgreSQL doesn't use these system-call a lots.
Especially, I think that checkpoint algorithm is very ugly..

Regards,
--
Mitsumasa KONDO
NTT Open Source Software Center