Re: PostgreSQL benchmarked on XFS vs ZFS vs btrfs vs ext4

On 15/09/11 02:46, Robert Treat wrote:
> Can you go into some more detail on how you set up ZFS on these systems?

I'm afraid my knowledge of ZFS is rather weak compared to the other
filesystems - all I really did was zpool create followed by zfs create,
using all the defaults.

The zpool was created against an LVM logical volume (which was the same
one used for all the filesystems measured in the tests). That LV was
itself part of a volume group that was striped over three disks (Western
Digital WD1003FBYX).

I'm happy to re-run the benchmark with different ZFS options if you can
suggest some. Unfortunately I can't easily bypass the LVM layer here, as
the disks involved are fully committed to the volume group and I don't
want to rebuild the machine.

Cheers,
Toby

> On Tue, Sep 13, 2011 at 10:56 PM, Andy Colson<andy(at)squeakycode(dot)net> wrote:
>> On 09/13/2011 08:15 PM, Toby Corkindale wrote:
>>>
>>> Hi,
>>> Some months ago, I ran some (probably naive) benchmarks looking at how
>>> pgbench performed on an identical system with differing filesystems. (on
>>> Linux).
>>>
>>> Since then the kernel-level version of ZFS became usable, and there have
>>> been improvements to btrfs, and no doubt various updates in the Linux kernel
>>> and PostgreSQL that should help performance.
>>>
>>> I ran the tests on Ubuntu 11.04 with Pg 9.0 first, then upgraded the
>>> system to Ubuntu 11.10 (beta) with Pg 9.1 and ran them again.
>>>
>>> The latter combination showed a considerable performance improvement
>>> overall - although I didn't investigate to find out whether this was due to
>>> kernel improvements, postgres improvements, or virtio improvements.
>>>
>>> The results are measured in transactions-per-second, with higher numbers
>>> being better.
>>>
>>> Results:
>>>
>>> ext4 (data=writeback,relatime):
>>> natty: 248
>>> oneiric: 297
>>>
>>> ext4 (data=writeback,relatime,nobarrier):
>>> natty: didn't test
>>> oneiric: 1409
>>>
>>> XFS (relatime):
>>> natty: didn't test
>>> oneiric: 171
>>>
>>> btrfs (relatime):
>>> natty: 61.5
>>> oneiric: 91
>>>
>>> btrfs (relatime,nodatacow):
>>> natty: didn't test
>>> oneiric: 128
>>>
>>> ZFS (defaults):
>>> natty: 171
>>> oneiric: 996
>>>
>>>
>>> Conclusion:
>>> Last time I ran these tests, xfs and ext4 pulled very similar results, and
>>> both were miles ahead of btrfs. This time around, ext4 has managed to get a
>>> significantly faster result than xfs.
>>>
>>> However we have a new contender - ZFS performed *extremely* well on the
>>> latest Ubuntu setup - achieving triple the performance of regular ext4!
>>> I'm not sure how it achieved this, and whether we're losing some kind of
>>> data protection (eg. like the "barrier" options in XFS and ext4).
>>> If ext4 has barriers disabled, it surpasses even ZFSs high score.
>>>
>>> Oddly, ZFS performed wildly differently on ubuntu 11.04 vs 11.10b. I can't
>>> explain this. Any ideas?