Re: file cloning in pg_upgrade and CREATE DATABASE

On Tue, Feb 20, 2018 at 10:00:04PM -0500, Peter Eisentraut wrote:
> Some new things have happened since then:
>
> - XFS has (optional) reflink support. This file system is probably more
> widely used than Btrfs.

Btrfs is still in development, there are I think no many people who
would use it in production.

> - Linux and glibc have a proper function to do this now.
>
> - APFS on macOS supports file cloning.

So copyfile() is only part of macos? I am not able to find references
in FreeBSD, NetBSD or OpenBSD, but I may be missing something.

> So altogether this feature will be more widely usable and less ugly to
> implement. Note, however, that you will currently need literally the
> latest glibc release, so it probably won't be accessible right now
> unless you are using Fedora 28 for example. (This is the
> copy_file_range() function that had us recently rename the same function
> in pg_rewind.)

For reference, Debian SID is using glibc 2.27. ArchLinux is still on
2.26.

> Some example measurements:
>
> 6 GB database, pg_upgrade unpatched 30 seconds, patched 3 seconds (XFS
> and APFS)

Interesting. I'll try to test that on an XFS partition and see if I can
see a difference. For now I have just read through the patch.

+#ifdef HAVE_COPYFILE
+ if (copyfile(fromfile, tofile, NULL,
+#ifdef COPYFILE_CLONE
+ COPYFILE_CLONE
+#else
+ COPYFILE_DATA
+#endif
+ ) < 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not copy file \"%s\" to \"%s\": %m", fromfile, tofile)));
+#else
copy_file(fromfile, tofile);
+#endif

Any backend-side callers of copy_file() would not benefit from
copyfile() on OSX. Shouldn't all that handling be inside copy_file(),
similarly to what your patch actually does for pg_upgrade? I think that
you should also consider fcopyfile() instead of copyfile() as it works
directly on the file descriptors and share the same error handling as
the others.
--
Michael