Re: ZFS filesystem - supported ?

We have some users of our software who have had a good experience with
postgresql on zfs/zol. Two features which have proved useful are the
native encryption (less fiddly than luks) and compression. Interestingly,
many of our users are stuck with quite old and slow disks. Using
compression (even together with encryption) on the slow disks gives quite a
significant performance boost. Trading cpu for disk bandwidth. Also they
often dont have infinite access to more disk, so the storage efficiency is
welcomed.

We are interested in the snapshots but a little wary of potential data
integrity issues.

We have a disturbance in our database structure (usually nightly) where
large tables are dropped and recreated. snapshots of a gradually
increasing size database probably work very well. I think these massive
deletions probably make the snapshots quite heavy. Also creating a
challenge for incremental backups, replication etc but that is another (not
quite unrelated) issue.

Regards
Bob

On Tue, 26 Oct 2021 at 01:18, Benedict Holland <benedict(dot)m(dot)holland(at)gmail(dot)com>
wrote:

> In my opinion, ext4 will solve any and all problems without a very deep
> understanding of file system architecture. In short, i would stick with
> ext4 unless you have a good reason not to. Maybe there is one. I have done
> this a long time and never thought twice about which file system should
> support my servers.
>
> On Mon, Oct 25, 2021, 6:01 PM Robert L Mathews <lists(at)tigertech(dot)com>
> wrote:
>
>> On 10/25/21 1:40 PM, Mladen Gogala wrote:
>> > This is probably not the place
>> > to discuss the inner workings of snapshots, but it is worth knowing
>> that
>> > snapshots drastically increase the IO rate on the file system - for
>> > every snapshot. That's where the slowness comes from.
>>
>> I have recent anecdotal experience of this. I experiment with using
>> Btrfs for a 32 TB backup system that has five 8 TB spinning disks.
>> There's an average of 8 MBps of writes scattered around the disks, which
>> isn't super high, obviously.
>>
>> The results were vaguely acceptable until I created a snapshot of it, at
>> which point it became completely unusable. Even having one snapshot
>> present caused hundreds of btrfs-related kernel threads to thrash in the
>> "D" state almost constantly, and it never stopped doing that even when
>> left for many hours.
>>
>> I then experimented with adding a bcache layer on top of Btrfs to see if
>> it would help. I added a 2 TB SSD using bcache, partitioned as 1900 GB
>> read cache and 100 GB write cache. It made very little difference and
>> was still unusable as soon as a snapshot was taken.
>>
>> I did play with the various btrfs and bcache tuning knobs quite a bit
>> and couldn't improve it.
>>
>> Since that test was a failure, I then decided to try the same setup with
>> OpenZFS on a lark, with the same set of disks in a "raidz" array, with
>> the 2 TB SSD as an l2arc read cache (no write cache). It easily handles
>> the same load, even with 72 hourly snapshots present, with the default
>> settings. I'm actually quite impressed with it.
>>
>> I'm sure that the RAID, snapshots and copy-on-write reduce the maximum
>> performance considerably, compared to ext4. But on the other hand, it
>> did provide the performance I expected to be possible given the setup.
>> Btrfs *definitely* didn't; I was surprised at how badly it performed.
>>
>> --
>> Robert L Mathews, Tiger Technologies, http://www.tigertech.net/
>>
>>
>>