Point-in-time recovery after failover

Hi

We are on Postgres 9.6 and we have primary/standby wal replication setup for HA.

For PITR, we have scheduled nightly base backup and hourly archive backups on the primary server.
https://www.postgresql.org/docs/9.6/static/continuous-archiving.html

I am trying to perform a point-in-time recovery after a failover has occurred.

For example:

1 am archive backup on P1
1:30 am base backup on P1 (primary) (timeline 5)
2 am archive backup on P1
3 am archive backup on P1
Etc..
1:49 pm failover P1 crashed, S1 got promoted. (timeline 6)
2 pm archive backup on S1
3 pm archive backup on S1

For example:
At 3:15pm something got corrupted and need to restore from backup to 1:54pm.

I extracted the base backups (tar files) to the data directory and extracted the all the archive backups (P1 and S1) to the archive directory.
Followed the steps from
Created recovery.conf with the following, but the recovery only recovers, to the point before the failover and comes online.

Example of my recovery.conf :
restore_command = 'cp /pg_backup/backup/archive/%f %p'
recovery_target_time = '2018-03-13 13:54:00'
recovery_target_inclusive = 'false'
recovery_target_timeline = '6' (I also tried revovery_target_timeline = 'lastest)

Is it possible to recovery to PITR after a failover using base backup from before failover and both archive backups from P1 and S1?

Here is the log output :

018-03-13 20:46:53 ACDT [154912]: [1-1] db=,user= app=,host= LOG: database system was interrupted; last known up at 2018-03-1
3 01:31:25 ACDT
2018-03-13 20:46:53 ACDT [154912]: [2-1] db=,user= app=,host= LOG: restored log file "00000006.history" from archive
cp: cannot stat '/pg_backup/backup/archive/00000007.history': No such file or directory
2018-03-13 20:46:53 ACDT [154912]: [3-1] db=,user= app=,host= LOG: starting point-in-time recovery to 2018-03-13 13:54:00+10:3
0
2018-03-13 20:46:53 ACDT [154912]: [4-1] db=,user= app=,host= LOG: restored log file "00000006.history" from archive
cp: cannot stat '/pg_backup/backup/archive/000000060000011100000087': No such file or directory
2018-03-13 20:46:53 ACDT [154912]: [5-1] db=,user= app=,host= LOG: restored log file "000000050000011100000087" from archive
cp: cannot stat '/pg_backup/backup/archive/00000005.history': No such file or directory
2018-03-13 20:46:53 ACDT [154912]: [6-1] db=,user= app=,host= LOG: redo starts at 111/87000028
2018-03-13 20:46:53 ACDT [154912]: [7-1] db=,user= app=,host= LOG: consistent recovery state reached at 111/870B4418
2018-03-13 20:46:53 ACDT [154908]: [3-1] db=,user= app=,host= LOG: database system is ready to accept read only connections
cp: cannot stat '/pg_backup/backup/archive/000000060000011100000088': No such file or directory
2018-03-13 20:46:53 ACDT [154912]: [8-1] db=,user= app=,host= LOG: restored log file "000000050000011100000088" from archive
cp: cannot stat '/pg_backup/backup/archive/000000060000011100000089': No such file or directory
2018-03-13 20:46:53 ACDT [154912]: [9-1] db=,user= app=,host= LOG: restored log file "000000050000011100000089" from archive
cp: cannot stat '/pg_backup/backup/archive/00000006000001110000008A': No such file or directory
2018-03-13 20:46:53 ACDT [154912]: [10-1] db=,user= app=,host= LOG: restored log file "00000005000001110000008A" from archive
cp: cannot stat '/pg_backup/backup/archive/00000006000001110000008B': No such file or directory
2018-03-13 20:46:53 ACDT [154912]: [11-1] db=,user= app=,host= LOG: restored log file "00000005000001110000008B" from archive
cp: cannot stat '/pg_backup/backup/archive/00000006000001110000008C': No such file or directory
2018-03-13 20:46:53 ACDT [154912]: [12-1] db=,user= app=,host= LOG: restored log file "00000005000001110000008C" from archive
cp: cannot stat '/pg_backup/backup/archive/00000006000001110000008D': No such file or directory
2018-03-13 20:46:53 ACDT [154912]: [13-1] db=,user= app=,host= LOG: restored log file "00000005000001110000008D" from archive
cp: cannot stat '/pg_backup/backup/archive/00000006000001110000008E': No such file or directory
2018-03-13 20:46:53 ACDT [154912]: [14-1] db=,user= app=,host= LOG: restored log file "00000005000001110000008E" from archive
cp: cannot stat '/pg_backup/backup/archive/00000006000001110000008F': No such file or directory
2018-03-13 20:46:54 ACDT [154912]: [15-1] db=,user= app=,host= LOG: restored log file "00000005000001110000008F" from archive
cp: cannot stat '/pg_backup/backup/archive/000000060000011100000090': No such file or directory
2018-03-13 20:46:54 ACDT [154912]: [16-1] db=,user= app=,host= LOG: restored log file "000000050000011100000090" from archive
cp: cannot stat '/pg_backup/backup/archive/000000060000011100000091': No such file or directory
2018-03-13 20:46:54 ACDT [154912]: [17-1] db=,user= app=,host= LOG: restored log file "000000050000011100000091" from archive
cp: cannot stat '/pg_backup/backup/archive/000000060000011100000092': No such file or directory
2018-03-13 20:46:54 ACDT [154912]: [18-1] db=,user= app=,host= LOG: restored log file "000000050000011100000092" from archive
cp: cannot stat '/pg_backup/backup/archive/000000060000011100000093': No such file or directory
2018-03-13 20:46:54 ACDT [154912]: [19-1] db=,user= app=,host= LOG: restored log file "000000050000011100000093" from archive
cp: cannot stat '/pg_backup/backup/archive/000000060000011100000094': No such file or directory
2018-03-13 20:46:54 ACDT [154912]: [20-1] db=,user= app=,host= LOG: restored log file "000000050000011100000094" from archive
cp: cannot stat '/pg_backup/backup/archive/000000060000011100000095': No such file or directory
2018-03-13 20:46:54 ACDT [154912]: [21-1] db=,user= app=,host= LOG: restored log file "000000050000011100000095" from archive
cp: cannot stat '/pg_backup/backup/archive/000000060000011100000096': No such file or directory
2018-03-13 20:46:54 ACDT [154912]: [22-1] db=,user= app=,host= LOG: restored log file "000000050000011100000096" from archive
cp: cannot stat '/pg_backup/backup/archive/000000060000011100000097': No such file or directory
2018-03-13 20:46:54 ACDT [154912]: [23-1] db=,user= app=,host= LOG: restored log file "000000050000011100000097" from archive
cp: cannot stat '/pg_backup/backup/archive/000000060000011100000098': No such file or directory
2018-03-13 20:46:55 ACDT [154912]: [24-1] db=,user= app=,host= LOG: restored log file "000000050000011100000098" from archive
cp: cannot stat '/pg_backup/backup/archive/000000060000011100000099': No such file or directory
2018-03-13 20:46:55 ACDT [154912]: [25-1] db=,user= app=,host= LOG: restored log file "000000050000011100000099" from archive
cp: cannot stat '/pg_backup/backup/archive/00000006000001110000009A': No such file or directory
2018-03-13 20:46:55 ACDT [154912]: [26-1] db=,user= app=,host= LOG: restored log file "00000005000001110000009A" from archive
cp: cannot stat '/pg_backup/backup/archive/00000006000001110000009B': No such file or directory
2018-03-13 20:46:55 ACDT [154912]: [27-1] db=,user= app=,host= LOG: restored log file "00000005000001110000009B" from archive
cp: cannot stat '/pg_backup/backup/archive/00000006000001110000009C': No such file or directory
2018-03-13 20:46:55 ACDT [154912]: [28-1] db=,user= app=,host= LOG: restored log file "00000005000001110000009C" from archive
cp: cannot stat '/pg_backup/backup/archive/00000006000001110000009D': No such file or directory
2018-03-13 20:46:55 ACDT [154912]: [29-1] db=,user= app=,host= LOG: restored log file "00000005000001110000009D" from archive
cp: cannot stat '/pg_backup/backup/archive/00000006000001110000009E': No such file or directory
2018-03-13 20:46:55 ACDT [154912]: [30-1] db=,user= app=,host= LOG: restored log file "00000005000001110000009E" from archive
cp: cannot stat '/pg_backup/backup/archive/00000006000001110000009F': No such file or directory
2018-03-13 20:46:55 ACDT [154912]: [31-1] db=,user= app=,host= LOG: restored log file "00000005000001110000009F" from archive
cp: cannot stat '/pg_backup/backup/archive/0000000600000111000000A0': No such file or directory
2018-03-13 20:46:55 ACDT [154912]: [32-1] db=,user= app=,host= LOG: restored log file "0000000500000111000000A0" from archive
cp: cannot stat '/pg_backup/backup/archive/0000000600000111000000A1': No such file or directory
2018-03-13 20:46:56 ACDT [154912]: [33-1] db=,user= app=,host= LOG: restored log file "0000000500000111000000A1" from archive
cp: cannot stat '/pg_backup/backup/archive/0000000600000111000000A2': No such file or directory
2018-03-13 20:46:56 ACDT [154912]: [34-1] db=,user= app=,host= LOG: restored log file "0000000500000111000000A2" from archive
cp: cannot stat '/pg_backup/backup/archive/0000000600000111000000A3': No such file or directory
2018-03-13 20:46:56 ACDT [154912]: [35-1] db=,user= app=,host= LOG: restored log file "0000000500000111000000A3" from archive
cp: cannot stat '/pg_backup/backup/archive/0000000600000111000000A4': No such file or directory
2018-03-13 20:46:56 ACDT [154912]: [36-1] db=,user= app=,host= LOG: restored log file "0000000500000111000000A4" from archive
cp: cannot stat '/pg_backup/backup/archive/0000000600000111000000A5': No such file or directory
2018-03-13 20:46:56 ACDT [154912]: [37-1] db=,user= app=,host= LOG: restored log file "0000000500000111000000A5" from archive
cp: cannot stat '/pg_backup/backup/archive/0000000600000111000000A6': No such file or directory
2018-03-13 20:46:56 ACDT [154912]: [38-1] db=,user= app=,host= LOG: restored log file "0000000500000111000000A6" from archive
cp: cannot stat '/pg_backup/backup/archive/0000000600000111000000A7': No such file or directory
2018-03-13 20:46:56 ACDT [154912]: [39-1] db=,user= app=,host= LOG: restored log file "0000000500000111000000A7" from archive
cp: cannot stat '/pg_backup/backup/archive/0000000600000111000000A8': No such file or directory
2018-03-13 20:46:56 ACDT [154912]: [40-1] db=,user= app=,host= LOG: restored log file "0000000500000111000000A8" from archive
cp: cannot stat '/pg_backup/backup/archive/0000000600000111000000A9': No such file or directory
2018-03-13 20:46:56 ACDT [154912]: [41-1] db=,user= app=,host= LOG: restored log file "0000000500000111000000A9" from archive
cp: cannot stat '/pg_backup/backup/archive/0000000600000111000000AA': No such file or directory
2018-03-13 20:46:57 ACDT [154912]: [42-1] db=,user= app=,host= LOG: restored log file "0000000500000111000000AA" from archive
cp: cannot stat '/pg_backup/backup/archive/0000000600000111000000AB': No such file or directory
2018-03-13 20:46:57 ACDT [154912]: [43-1] db=,user= app=,host= LOG: restored log file "0000000500000111000000AB" from archive
cp: cannot stat '/pg_backup/backup/archive/0000000600000111000000AC': No such file or directory
2018-03-13 20:46:57 ACDT [154912]: [44-1] db=,user= app=,host= LOG: restored log file "0000000500000111000000AC" from archive
cp: cannot stat '/pg_backup/backup/archive/0000000600000111000000AD': No such file or directory
2018-03-13 20:46:57 ACDT [154912]: [45-1] db=,user= app=,host= LOG: restored log file "0000000500000111000000AD" from archive
cp: cannot stat '/pg_backup/backup/archive/0000000600000111000000AE': No such file or directory
cp: cannot stat '/pg_backup/backup/archive/0000000500000111000000AE': No such file or directory
2018-03-13 20:46:57 ACDT [154912]: [46-1] db=,user= app=,host= LOG: redo done at 111/ADFFE160
2018-03-13 20:46:57 ACDT [154912]: [47-1] db=,user= app=,host= LOG: last completed transaction was at log time 2018-03-13 13:4
0:02.36347+10:30
cp: cannot stat '/pg_backup/backup/archive/0000000600000111000000AD': No such file or directory
2018-03-13 20:46:57 ACDT [154912]: [48-1] db=,user= app=,host= LOG: restored log file "0000000500000111000000AD" from archive
cp: cannot stat '/pg_backup/backup/archive/00000007.history': No such file or directory
2018-03-13 20:46:57 ACDT [154912]: [49-1] db=,user= app=,host= LOG: selected new timeline ID: 7
2018-03-13 20:46:57 ACDT [154912]: [50-1] db=,user= app=,host= LOG: restored log file "00000006.history" from archive
2018-03-13 20:46:57 ACDT [154912]: [51-1] db=,user= app=,host= LOG: archive recovery complete
2018-03-13 20:46:57 ACDT [154919]: [1-1] db=,user= app=,host= LOG: checkpoint starting: end-of-recovery immediate wait
2018-03-13 20:46:59 ACDT [154919]: [2-1] db=,user= app=,host= LOG: checkpoint complete: wrote 53374 buffers (0.6%); 0 transact
ion log file(s) added, 0 removed, 32 recycled; write=1.140 s, sync=0.019 s, total=1.224 s; sync files=68, longest=0.013 s, aver
age=0.000 s; distance=638968 kB, estimate=638968 kB
2018-03-13 20:46:59 ACDT [154912]: [52-1] db=,user= app=,host= LOG: MultiXact member wraparound protections are now enabled
2018-03-13 20:46:59 ACDT [154908]: [4-1] db=,user= app=,host= LOG: database system is ready to accept connections

Here is what is in the archive directory after extracting from the archive backups and after database comes online:

-rw-------. 1 postgres postgres 16777216 Mar 12 21:13 000000050000011100000075
-rw-------. 1 postgres postgres 16777216 Mar 12 21:30 000000050000011100000076
-rw-------. 1 postgres postgres 16777216 Mar 12 21:47 000000050000011100000077
-rw-------. 1 postgres postgres 16777216 Mar 12 22:05 000000050000011100000078
-rw-------. 1 postgres postgres 16777216 Mar 12 22:22 000000050000011100000079
-rw-------. 1 postgres postgres 16777216 Mar 12 22:38 00000005000001110000007A
-rw-------. 1 postgres postgres 16777216 Mar 12 22:56 00000005000001110000007B
-rw-------. 1 postgres postgres 16777216 Mar 12 23:13 00000005000001110000007C
-rw-------. 1 postgres postgres 16777216 Mar 12 23:30 00000005000001110000007D
-rw-------. 1 postgres postgres 16777216 Mar 12 23:36 00000005000001110000007E
-rw-------. 1 postgres postgres 16777216 Mar 12 23:45 00000005000001110000007F
-rw-------. 1 postgres postgres 16777216 Mar 13 00:02 000000050000011100000080
-rw-------. 1 postgres postgres 16777216 Mar 13 00:20 000000050000011100000081
-rw-------. 1 postgres postgres 16777216 Mar 13 00:36 000000050000011100000082
-rw-------. 1 postgres postgres 16777216 Mar 13 00:53 000000050000011100000083
-rw-------. 1 postgres postgres 16777216 Mar 13 01:10 000000050000011100000084
-rw-------. 1 postgres postgres 16777216 Mar 13 01:27 000000050000011100000085
-rw-------. 1 postgres postgres 16777216 Mar 13 01:30 000000050000011100000086
-rw-------. 1 postgres postgres 16777216 Mar 13 01:31 000000050000011100000087
-rw-------. 1 postgres postgres 307 Mar 13 01:31 000000050000011100000087.00000028.backup
-rw-------. 1 postgres postgres 16777216 Mar 13 01:48 000000050000011100000088
-rw-------. 1 postgres postgres 16777216 Mar 13 02:05 000000050000011100000089
-rw-------. 1 postgres postgres 16777216 Mar 13 02:22 00000005000001110000008A
-rw-------. 1 postgres postgres 16777216 Mar 13 02:39 00000005000001110000008B
-rw-------. 1 postgres postgres 16777216 Mar 13 02:55 00000005000001110000008C
-rw-------. 1 postgres postgres 16777216 Mar 13 03:12 00000005000001110000008D
-rw-------. 1 postgres postgres 16777216 Mar 13 03:30 00000005000001110000008E
-rw-------. 1 postgres postgres 16777216 Mar 13 03:47 00000005000001110000008F
-rw-------. 1 postgres postgres 16777216 Mar 13 04:04 000000050000011100000090
-rw-------. 1 postgres postgres 16777216 Mar 13 04:21 000000050000011100000091
-rw-------. 1 postgres postgres 16777216 Mar 13 04:38 000000050000011100000092
-rw-------. 1 postgres postgres 16777216 Mar 13 04:55 000000050000011100000093
-rw-------. 1 postgres postgres 16777216 Mar 13 05:12 000000050000011100000094
-rw-------. 1 postgres postgres 16777216 Mar 13 05:28 000000050000011100000095
-rw-------. 1 postgres postgres 16777216 Mar 13 05:46 000000050000011100000096
-rw-------. 1 postgres postgres 16777216 Mar 13 06:02 000000050000011100000097
-rw-------. 1 postgres postgres 16777216 Mar 13 06:19 000000050000011100000098
-rw-------. 1 postgres postgres 16777216 Mar 13 06:36 000000050000011100000099
-rw-------. 1 postgres postgres 16777216 Mar 13 06:53 00000005000001110000009A
-rw-------. 1 postgres postgres 16777216 Mar 13 07:11 00000005000001110000009B
-rw-------. 1 postgres postgres 16777216 Mar 13 07:29 00000005000001110000009C
-rw-------. 1 postgres postgres 16777216 Mar 13 07:47 00000005000001110000009D
-rw-------. 1 postgres postgres 16777216 Mar 13 08:02 00000005000001110000009E
-rw-------. 1 postgres postgres 16777216 Mar 13 08:02 00000005000001110000009F
-rw-------. 1 postgres postgres 16777216 Mar 13 08:19 0000000500000111000000A0
-rw-------. 1 postgres postgres 16777216 Mar 13 08:36 0000000500000111000000A1
-rw-------. 1 postgres postgres 16777216 Mar 13 08:55 0000000500000111000000A2
-rw-------. 1 postgres postgres 16777216 Mar 13 09:13 0000000500000111000000A3
-rw-------. 1 postgres postgres 16777216 Mar 13 09:15 0000000500000111000000A4
-rw-------. 1 postgres postgres 16777216 Mar 13 09:15 0000000500000111000000A5
-rw-------. 1 postgres postgres 16777216 Mar 13 09:17 0000000500000111000000A6
-rw-------. 1 postgres postgres 16777216 Mar 13 10:03 0000000500000111000000A7
-rw-------. 1 postgres postgres 16777216 Mar 13 10:15 0000000500000111000000A8
-rw-------. 1 postgres postgres 16777216 Mar 13 13:40 0000000500000111000000A9
-rw-------. 1 postgres postgres 16777216 Mar 13 13:40 0000000500000111000000AA
-rw-------. 1 postgres postgres 16777216 Mar 13 13:40 0000000500000111000000AB
-rw-------. 1 postgres postgres 16777216 Mar 13 13:40 0000000500000111000000AC
-rw-------. 1 postgres postgres 16777216 Mar 13 13:40 0000000500000111000000AD
-rw-------. 1 postgres postgres 16777216 Mar 13 13:48 0000000500000111000000B1.partial
-rw-------. 1 postgres postgres 16777216 Mar 13 13:49 0000000600000111000000B1
-rw-------. 1 postgres postgres 16777216 Mar 13 14:04 0000000600000111000000B2
-rw-------. 1 postgres postgres 221 Mar 13 13:48 00000006.history
-rw-------. 1 postgres postgres 16777216 Mar 13 20:48 0000000700000111000000AD
-rw-------. 1 postgres postgres 271 Mar 13 20:46 00000007.history

Dylan