Re: a question about Direct I/O and double buffering

On Apr 5, 2007, at 1:22 PM, Xiaoning Ding wrote:

> Erik Jones wrote:
>> On Apr 5, 2007, at 12:09 PM, Xiaoning Ding wrote:
>>> Hi,
>>>
>>> A page may be double buffered in PG's buffer pool and in OS's
>>> buffer cache.
>>> Other DBMS like DB2 and Oracle has provided Direct I/O option to
>>> eliminate
>>> double buffering. I noticed there were discusses on the list. But
>>> I can not find similar option in PG. Does PG support direct I/O now?
>>>
>>> The tuning guide of PG usually recommends a small shared buffer pool
>>> (compared
>>> to the size of physical memory). I think it is to avoid
>>> swapping. If
>>> there were
>>> swapping, OS kernel may swap out some pages in PG's buffer pool
>>> even PG
>>> want to keep them in memory. i.e. PG would loose full control over
>>> buffer pool.
>>> A large buffer pool is not good because it may
>>> 1. cause more pages double buffered, and thus decrease the
>>> efficiency of
>>> buffer
>>> cache and buffer pool.
>>> 2. may cause swapping.
>>> Am I right?
>>>
>>> If PG's buffer pool is small compared with physical memory, can I
>>> say
>>> that the
>>> hit ratio of PG's buffer pool is not so meaningful because most
>>> misses
>>> can be
>>> satisfied by OS Kernel's buffer cache?
>>>
>>> Thanks!
>> To the best of my knowledge, Postgres itself does not have a
>> direct IO option (although it would be a good addition). So, in
>> order to use direct IO with postgres you'll need to consult your
>> filesystem docs for how to set the forcedirectio mount option. I
>> believe it can be set dynamically, but if you want it to be
>> permanent you'll to add it to your fstab/vfstab file.
>
> I use Linux. It supports direct I/O on a per-file basis only. To
> bypass OS buffer cache,
> files should be opened with O_DIRECT option. I afraid that I have
> to modify PG.
>
> Xiaoning

Looks like it. I just did a cursory search of the archives and it
seems that others have looked at this before so you'll probably want
to start there if your up to it.

erik jones <erik(at)myemma(dot)com>
software developer
615-296-0838
emma(r)