Re: [patch] pg_copy - a command for reliable WAL archiving

On 8/19/14 9:35 AM, MauMau wrote:
> pg_copy is for copying a file reliably by syncing. If sync is not
> necessary, people can use cp/copy.

I'm getting mixed messages from this thread.

I think there could be a fair amount of support for a new tool that can
serve as a universal plug-and-play archive_command, with a variety of
options, such as fsync yes/no, fadvise yes/no, direct-io[*] yes/no,
atomic copy yes/no, allow overwrite yes/no, compression yes/no. That
would reduce the need for users to compose adventurous shell commands,
and it would set out the various options more clearly.

This is not that. This is cp+fsync with a hardcoded fadvise policy and
optional direct-io. That is a valid problem to address, but it is one
of many. On the other hand, I fear that the addition of this
single-and-a-half-purpose tool would make the overall landscape more
complicated than it already is. Since it's in the examples, people will
probably use it, even if they don't need to or shouldn't. And not
recommending it for the restore_command is also confusing.

Another example of how confusing all of this is: On Windows, the copy
command by default doesn't overwrite files, which is what we want
(usually). The patch changes those instances of copy to pg_copy, but it
doesn't have that behavior. Should the examples by changed to do a test
&& pg_copy on Windows (what's the Windows shell syntax for that?), or
should pg_copy have an option to not overwrite a file? How do you then
avoid inconsistencies with the Unix behavior? Or what if you want fsync
but allow overwriting on Windows?

On the technical side, I think if you fsync a new file, you also need to
fsync the directory, to make sure the file is certain to be visible
after a crash.

[*] I keep reading "directio" as a typo of "direction", so please
consider putting a hyphen or underscore in the option and variable
names. ;-)