Re: Reliable WAL file shipping over unreliable network

On 07/03/18 08:31, David Steele wrote:

> On 3/6/18 2:00 PM, Stephen Frost wrote:
>> * Rui DeSousa (rui(dot)desousa(at)icloud(dot)com) wrote:
>>>> On Mar 5, 2018, at 10:02 AM, Stephen Frost <sfrost(at)snowman(dot)net> wrote:
>>>> It doesn’t- but please don't encourage partial solutions which have ver clear issues.
>>> Then problem is there are no good base utilities that is useful with archive_command; unless you’re writing directly to an NFS mount or a tape library. Even Barman recommends rsync with the archive_command; if you are unable to use pg_receivexlog solution. There are countless Postgres documentation out there that recommends use of rsync with the archive_command.
>>>
>>> Here a solution that will fsync() file on the other end.
>> I'd encourage you to consider submitting this to a commitfest to get
>> feedback on it, as a proper patch to add such a utlity to PG and which
>> we could use in our documentation instead of saying "cp", which is
>> pretty terrible.
> +1. I believe there's been talk of a pgcopy utility before but it never
> went anywhere. I think it would be worth it just to have a decent
> example in the docs.
>
+1. I think if there is a strong feeling that the current docs are
suggesting things that shouldn't actually be done, then we should
provide an alternative in the distribution itself.

Re Pgbackrest (and other backup tools that various Postgres companies
have written) - while these may be great (Pgbackrest looks pretty good
for instance), they don't come with the Postgres package itself, and our
docs suggest something else. Hence discussion threads like this one!

The added (new) issue that there may be some unusual bug with rsync +
quotas and ZFS on BSD (at least) gives a bit more weight to this idea.

regards
Mark