Re: [HACKERS] New wal_sync_method for Darwin?

Attached is an updated version of your Darwin Fsync-writethrough patch.
I did much more restructuring because we do fsync in more places than
just WAL (like at checkpoints), and we should follow the WAL
write-through in those places too if that method is used for fsync of
WAL, rather than the normal fsync without the force-writethrough.

I restructured the code so now pg_fsync does writethrough or not based
on the wal setting. My feeling is you need WAL writethrough, you need
it for other fsyncs as well. Write-through was added in an 8.0.X
subrelease, and because it was a subrelease, the patch did a minimal
amount of code change. This change is the restructuring that makes
sense for 8.1.

---------------------------------------------------------------------------

Chris Campbell wrote:
> Thanks, Bruce! Here's the patch (would you rather patches be attached
> files, or inline?).
>
> Some explanation. In the current code, there is no "fsync" on Windows,
> only "fsync_writethrough". But, internally, it behaves identically to
> "fsync". The only special case was for the GUC strings.
>
> The patch changes it thusly: since "fsync" and "fsync_writethrough"
> cannot be identical on Darwin, I added a new constant for
> SYNC_METHOD_FSYNC_WRITE_THROUGH (3). A GUC value of
> "fsync_writethrough" sets fsync_method to this new constant.
>
> On Windows, the fsync_writethrough case falls through to the fsync case
> (which simply runs fsync). On Darwin, the fsync_writethrough case
> executes fcntl(openLogFile, F_FULLFSYNC, 0), which does everything that
> fsync does, and then flushes the disk's write cache. The code does not
> need to call fsync directly, just fcntl.
>
> The performance of a script that executes 11,000 INSERT statements
> (outside a transaction) is pretty slow, since it requires flushing the
> disk write cache after EVERY query (transaction commit). I waited about
> 15 minutes before I just killed it. But if you do all the inserts
> inside a transaction, then I only pay the cache flush price once at
> transaction commit, and it's quite usable.
>
> I asked around on IRC, and the only place where flushing the disk cache
> is important is when syncing a WAL file. So I didn't change the
> behavior of fsync() globally for the code...just for WAL syncs.
>
> I added a new #define called HAVE_FSYNC_WRITE_THROUGH, which is defined
> only by the win32.h and darwin.h port headers. This controls the
> availability of the "fsync_writethrough" GUC setting. Thanks to the
> port headers, no configure test is needed.
>
> Thanks! Let me know what you think of the patch, and tell me what else
> I can do.
>
> - Chris
>
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073