Rsync to a recovering streaming replica?

From: Igor Polishchuk <ora4dba(at)gmail(dot)com>
To: pgsql-general(at)postgresql(dot)org
Subject: Rsync to a recovering streaming replica?
Date: 2017-09-27 17:56:27
Message-ID: 7DF443A3-A63A-4369-8C41-50590F7EE5FA@gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hello,
I have a multi-terabyte streaming replica on a bysy database. When I set it up, repetative rsyncs take at least 6 hours each.
So, when I start the replica, it begins streaming, but it is many hours behind right from the start. It is working for hours, and cannot reach a consistent state
so the database is not getting opened for queries. I have plenty of WAL files available in the master’s pg_xlog, so the replica never uses archived logs.
A question:
Should I be able to run one more rsync from the master to my replica while it is streaming?
The idea is to overcome the throughput limit imposed by a single recovery process on the replica and allow to catch up quicker.
I remember doing it many years ago on Pg 8.4, and also heard from other people doing it. In all cases, it seamed working.
I’m just not sure if there is no high risk of introducing some hidden data corruption, which I may not notice for a while on such a huge database.
Any educated opinions on the subject here?

Thank you
Igor Polishchuk

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Igor Polishchuk 2017-09-27 17:59:34 Re: Rsync to a recovering streaming replica?
Previous Message David G. Johnston 2017-09-27 16:59:40 Re: Catching errors inside a LOOP is causing performance issues