Re: Reliable WAL file shipping over unreliable network

>> Another problem with rsync might be that it does not know if a WAL file
>> on the source side is growing (e.g. being written) and it might start
>> copy that before it is fully flushed to disk. I see this as a big
>> problem, but I don't have experience.
> You use rsync in the archive command, not by itself. Postgres does not try to copy the file until it is ready.
Well, that will be a problem because postgres is running in an isolated
docker container. But that is a totally different problem. I can see
that if rsync is used in the archive command, then postgres will retry
unsuccessful transfers on the source side.
>
>> E.g. move the file into the archive dir only after it has been fully
>> copied to the destination volume. But then we still have the problem of
>> partially written files on the slave side.
> No, you do not have that problem at all.
Can you please explain how postres on the replication slave can
determine if the file in the archive dir is completely flushed to the disk?
>
> Given an unreliable network, you probably do want to use --partial
>
--partial can resume interrupted transfers. But on the slave side, there
will be partially transferred files in the archive dir, maybe for a long
time.

As far as I understand, the restore command will not (and cannot)
examine of the source file is completely copied. It will just be called,
and if there is a file with the expected name, then it will copy it into
pg_wal.

Maybe I'm missing a key concept, but I still don't see how it works.

  L