Re: pg_combinebackup --copy-file-range

On Sun, Mar 31, 2024 at 1:37 PM Tomas Vondra
<tomas(dot)vondra(at)enterprisedb(dot)com> wrote:
> So I decided to take a stab at Thomas' idea, i.e. reading the data to
> ...
> I'll see how this works on EXT4/ZFS next ...

Wow, very cool! A couple of very quick thoughts/notes:

ZFS: the open source version only gained per-file block cloning in
2.2, so if you're on an older release I expect copy_file_range() to
work but not really do the magic thing. On the FreeBSD version you
also have to turn cloning on with a sysctl because people were worried
about bugs in early versions so by default you still get actual
copying, not sure if you need something like that on the Linux
version... (Obviously ZFS is always completely COW-based, but before
the new block cloning stuff it could only share blocks by its own
magic deduplication if enabled, or by cloning/snapshotting a whole
dataset/mountpoint; there wasn't a way to control it explicitly like
this.)

Alignment: block sharing on any fs requires it. I haven't re-checked
recently but IIRC the incremental file format might have a
non-block-sized header? That means that if you copy_file_range() from
both the older backup and also the incremental backup, only the former
will share blocks, and the latter will silently be done by copying to
newly allocated blocks. If that's still true, I'm not sure how hard
it would be to tweak the format to inject some padding and to make
sure that there isn't any extra header before each block.