Running from 9.6 backups sometimes fails with fatal error

Hi!

I need help with investigation what happened here. I have two different master servers with standby,
version: PostgreSQL 9.6.10 on x86_64-pc-linux-gnu (Debian 9.6.10-1.pgdg90+1), compiled by gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516, 64-bit

and I create backups with pg_basebackup from theirs standbys, but sometimes backup check fails,
it just cannot start, for example:

server 1:
2019-02-18 14:18:51 MSK [11901-2] LOG: checkpoint starting: end-of-recovery immediate wait
2019-02-18 14:18:51 MSK [11901-3] DEBUG: performing replication slot checkpoint
2019-02-18 14:18:51 MSK [11901-4] LOG: request to flush past end of generated WAL; request 44B/7E5DAB28, currpos 44B/7305B560
2019-02-18 14:18:51 MSK [11901-5] CONTEXT: writing block 0 of relation base/16506/16891_vm
2019-02-18 14:18:51 MSK [11901-6] ERROR: xlog flush request 44B/7E5DAB28 is not satisfied --- flushed only to 44B/7305B560
2019-02-18 14:18:51 MSK [11901-7] CONTEXT: writing block 0 of relation base/16506/16891_vm
2019-02-18 14:18:51 MSK [11858-110] FATAL: checkpoint request failed

server 2:
2019-02-07 16:07:40 MSK [7911-37] LOG: checkpoint starting: end-of-recovery immediate wait
2019-02-07 16:07:40 MSK [7911-38] DEBUG: performing replication slot checkpoint
2019-02-07 16:07:41 MSK [7911-39] LOG: request to flush past end of generated WAL; request 1CB/8E94EE90, currpos 1CB/8A195210
2019-02-07 16:07:41 MSK [7911-40] CONTEXT: writing block 0 of relation base/18413/18529_vm
2019-02-07 16:07:41 MSK [7911-41] ERROR: xlog flush request 1CB/8E94EE90 is not satisfied --- flushed only to 1CB/8A195210
2019-02-07 16:07:41 MSK [7911-42] CONTEXT: writing block 0 of relation base/18413/18529_vm
2019-02-07 16:07:41 MSK [7868-665] FATAL: checkpoint request failed

backup checked with: PostgreSQL 9.6.11 on x86_64-pc-linux-gnu (Debian 9.6.11-1.pgdg90+1), compiled by gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516, 64-bit

recovery.conf:
restore_command = '/usr/local/bin/restore_cmd /mnt/wals %f %p'
recovery_target = 'immediate'

postgresql.conf:
wal_log_hints = on

How can I debug this?

--
Sergey Burladyan