Re: O_DIRECT support for Windows

From: "Takayuki Tsunakawa" <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com>
To: "Magnus Hagander" <magnus(at)hagander(dot)net>
Cc: "ITAGAKI Takahiro" <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp>, <pgsql-patches(at)postgresql(dot)org>
Subject: Re: O_DIRECT support for Windows
Date: 2007-01-17 00:52:53
Message-ID: 00d101c739d1$d1e7dc90$19527c0a@OPERAO
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-patches

Hello, Magnus-san, Itagaki-san

From: "Magnus Hagander" <magnus(at)hagander(dot)net>
>> I think many people can benefit from Itagaki-san's proposal, and
>> NO_BUFFERING should be default. Isn't it very rare that disks with
>> sector size larger than 8KB are used?
>
> Definitly very rare.
>
>
>> Providing a way (such as
>> wal_sync_method) to avoid NO_BUFFERING is sufficient for people in
>> rare environments. Or, by determining the sector size with
>> GetDiskFreeSpaceEx(), we could auto-switch to not using
NO_BUFFERING
>> when the sector size is larger than 8KB.
>
> I think the second one is better.

Thank you for agreeing. Then, I hope Itagaki-san's patch will be
accepted when the following treatments are added to the patch and some
performance report is delivered.

1. On Windows, O_DIRECT (and O_SYNC?) is default for WAL.
2. Auto-switch to not using O_DIRECT if the sector size is larger than
8KB when the server starts.

> A quick google shows some inconclusive results :-)BUt look at for
> example:
>
http://groups.google.se/group/microsoft.public.sqlserver.server/tree/browse_frm/thread/d3288d3b43338b47/ff5e825dd02faff4?rnum=1&hl=en&q=ntfs+sector+size&_done=%2Fgroup%2Fmicrosoft.public.sqlserver.server%2Fbrowse_frm%2Fthread%2Fd3288d3b43338b47%2Fff5e825dd02faff4%3Ftvc%3D1%26q%3Dntfs+sector+size%26hl%3Den%26#doc_4556b64132b3baa7
>
> This seems to indicate that *Windows* supports sector sizes >4K, but
SQL
> Server doesn't. But again, it could be a mixup between cluster and
> sector size...

This is interesting. I've never seen systems with a sector size
larger than 4KB, too. On IBM zSeries (which is a mainframe running
Linux), DASD (direct attached storage device) is usually used as a
hard disk. The sector size of DASD is 4KB. So, the current
implementation of PostgreSQL which assumes 8KB sector size is
practically sufficient.
Delivering an intuitive error message like SQL Server is one way when
PostgreSQL encounters devices with a larger sector size than is
supported. However, as you say, auto-switching to not using
NO_BUFFERING is kinder to users.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Takayuki Tsunakawa 2007-01-17 01:56:05 Re: Idea for fixing the Windows fsync problem
Previous Message Tom Lane 2007-01-17 00:18:14 Re: Idea for fixing the Windows fsync problem

Browse pgsql-patches by date

  From Date Subject
Next Message Chuck McDevitt 2007-01-17 07:09:29 Re: O_DIRECT support for Windows
Previous Message Neil Conway 2007-01-16 21:43:33 Re: vcbuild updates