RE: Changing default value of wal_sync_method to open_datasync on Linux

From: "Tsunakawa, Takayuki" <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com>
To: 'Mark Kirkwood' <mark(dot)kirkwood(at)catalyst(dot)net(dot)nz>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: RE: Changing default value of wal_sync_method to open_datasync on Linux
Date: 2018-02-20 01:56:17
Message-ID: 0A3221C70F24FB45833433255569204D1F8CE01E@G01JPEXMBYT05
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

From: Mark Kirkwood [mailto:mark(dot)kirkwood(at)catalyst(dot)net(dot)nz]
> I think the use of 'nobarrier' is probably disabling most/all reliable
> writing to the devices. What do the numbers look like if use remove this
> option?

Disabling the filesystem barrier is a valid tuning method as the PG manual says:

https://www.postgresql.org/docs/devel/static/wal-reliability.html

[Excerpt]
Recent SATA drives (those following ATAPI-6 or later) offer a drive cache flush command (FLUSH CACHE EXT), while SCSI drives have long supported a similar command SYNCHRONIZE CACHE. These commands are not directly accessible to PostgreSQL, but some file systems (e.g., ZFS, ext4) can use them to flush data to the platters on write-back-enabled drives. Unfortunately, such file systems behave suboptimally when combined with battery-backup unit (BBU) disk controllers. In such setups, the synchronize command forces all data from the controller cache to the disks, eliminating much of the benefit of the BBU. You can run the pg_test_fsync program to see if you are affected. If you are affected, the performance benefits of the BBU can be regained by turning off write barriers in the file system or reconfiguring the disk controller, if that is an option. If write barriers are turned off, make sure the battery remains functional; a faulty battery can potentially lead to data loss. Hopefully file system and disk controller designers will eventually address this suboptimal behavior.

I removed nobarrier mount option on a VM with HDD. pgbench throughput decreased about 30% to 3550 tps, but the relative difference between fdatasync and open_datasync is similar. I cannot disable nowritebarrier on the bare metal with SSD for now, as other developers and test teams are using it.

Regards
Takayuki Tsunakawa

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Alvaro Herrera 2018-02-20 02:03:53 Re: Server crash in pg_replication_slot_advance function
Previous Message Masahiko Sawada 2018-02-20 01:47:39 Typo in procarray.c