Re: Pg_rewind cannot load history wal

From: Abhinav Mehta <abhinav(at)metarain(dot)com>
To: Richard Schmidt <Richard(dot)Schmidt(at)metservice(dot)com>
Cc: "pgsql-general(at)lists(dot)postgresql(dot)org" <pgsql-general(at)lists(dot)postgresql(dot)org>
Subject: Re: Pg_rewind cannot load history wal
Date: 2018-08-02 05:00:33
Message-ID: 2977F427-BDA5-45A7-B140-F3EAA1D8E422@metarain.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Whenever you do switch-over, postgres-wal creates a new timeline, which simplifies managing PITR process.

During switch-over(promoting B as master) you had some delta records written to A, that’s where it causes this timeline issue.

Now since A had some delta records, it can’t replicate from B and hence you are getting that issue.

Now once your master A can’t become slave of B.
— A

> On 02-Aug-2018, at 2:39 AM, Richard Schmidt <Richard(dot)Schmidt(at)metservice(dot)com> wrote:
>
> We have been struggling to get pr_rewind to work. In desperation we have been trying to make the use case as simple as possibleJ
>
> We have two databases servers running Postgres 10 on two different machine in the normal Primary/Standby configuration.
> Both machines write their WAL archive logs to the same shared drive (called /ice_dev/wal_archive).
> The configuration has the following terms
> archive_mode = always
> archive_command = 'test ! -f /ice-dev/wal_archive/%f && cp %p /ice-dev/wal_archive/%f'
> full_page_writes = on
> wal_log_hints = on
> Checksums are enabled
>
> Our procedure that runs on machine A and B is as follows:
>
> Build new databases on A and B, and configure A as Primary and B as Standby databases.
> Make some changes to the A (the primary) and check that they are replicated to the B (the standby)
> Promote B to be the new primary
> Switch of the A (the original primary)
> Add the replication slot to B (the new primary) for A (soon to be standby)
> Add a recovery.conf to A (soon to be standby). File contains recovery_target_timeline = 'latest' and restore_command = 'cp /ice-dev/wal_archive/%f "%p"
> Run pg_rewind on A – this appears to work as it returns the message ‘source and target cluster are on the same timeline no rewind required’;
> Start up server A (now a slave)
>
> At this point A is in a read only mode but not replicating. Its logs contain the following repeating message
>
> 2018-08-01 20:30:58 UTC [7257]: [1] user=,db=,app=,client= FATAL: could not start WAL streaming: ERROR: requested starting point 0/6000000 on timeline 1 is not in this server's history
> DETAIL: This server's history forked from timeline 1 at 0/57639D0.
> cp: cannot stat ‘/ice-dev/wal_archive/00000002.history’: No such file or directory
> cp: cannot stat ‘/ice-dev/wal_archive/00000003.history’: No such file or directory
> cp: cannot stat ‘/ice-dev/wal_archive/00000002.history’: No such file or directory
> 2018-08-01 20:30:58 UTC [6840]: [48] user=,db=,app=,client= LOG: new timeline 2 forked off current database system timeline 1 before current recovery point 0/6000098
> cp: cannot stat ‘/ice-dev/wal_archive/000000010000000000000006’: No such file or directory
>
> We can see the 00000002.history file in B’s wal directory…..but it never appears in the wal_archive directory – not even if we issue a checkout or even restart the server.
> 00000003.history does not appear to exist on either of the machines.
>
> Any ideas what we are doing wrong?
> Thanks. Richard
>
>
>
>
>
>
>
> This email and any attachments may contain confidential information. If you are not the intended recipient, your use or communication of the information is strictly prohibited. If you have received this message in error please notify MetService immediately.
>

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Alexandru Lazarev 2018-08-02 05:53:15 Who and How is responsible for released installations packages and 3rd party packs? (e.g. on https://yum.postgresql.org/9.6/redhat/rhel-7.3-x86_64/)
Previous Message Igor Korot 2018-08-02 01:08:19 Problem building libpq