Re: pg_basebackup connection closed unexpectedly...

=?UTF-8?Q?Mladen_Marinovi=C4=87?= <mladen(dot)marinovic(at)kset(dot)org> writes:
> Recently I am having some strange problems with pg_basebackup. About once a
> week the backup process ends with an error message like this:
> 2020-02-11 23:25:40 UTC [25790]: [1-1] user=replicator,db=[unknown] LOG:
> could not send data to client: Connection reset by peer

Hmmm ....

> The problem started occurring after a hardware (RAM + SSD) upgrade and an
> OS Upgrade to Ubuntu 18.04. Both the server and backup process run in
> separate docker containers on the same machine. This happens randomly on
> multiple servers with the same configuration and it is probably not
> hardware related. Also, this happens evenly on 9.4 and 9.6, and using the
> same docker images that worked flawlessly on the previous installation.
> I have been investigating the issue for at least a month and found no
> problems in any log or metric before or after the event. I suspect that
> this is related to some OS/docker parameter that is not well configured.

How long does the backup run before failing? If the connection were going
between different machines my suspicions would lean toward a network
timeout. That seems somewhat unlikely in this configuration, but you
never know.

> Would increasing the database log level give me any more info about what
> caused the connection to close?

Nope, not directly. It might be useful to figure out whether data
transfer continues full throttle right up until the connection drop,
or whether it stops sooner (and then there's some sort of timeout
before the error occurs).

regards, tom lane