Re: Standby is not removing restored WAL segments

On 16-09-2014 16:27, Alexey Klyukin wrote:
> On Mon, Sep 15, 2014 at 5:33 PM, Guillaume Lelarge
> <guillaume(at)lelarge(dot)info> wrote:
>
>> I have no direct answer for you, but we seem to have the same issue for two
>> of our customers. We are on 9.2.8 on one of them. Do you know if you have
>> the .ready related files in the archive_status directory? are they old WAL
>> files? can you tell us their names?
> We don't have their names anymore, but the stalled WAL segments were
> from the same
> timeline as the up-to-date ones and were removed after we did a restart.
>
> As for the ready files, there are quite a lot of them:
>
>
> pg_xlog/archive_status$ ls -la
> total 68
> drwx------ 2 postgres postgres 28672 Sep 16 12:46 .
> drwxr-xr-x 3 postgres postgres 20480 Sep 16 12:41 ..
> -rw------- 1 postgres postgres 0 Apr 10 16:47 0000000100006C17000000D3.ready
> -rw------- 1 postgres postgres 0 Apr 10 16:47 0000000100006C17000000D4.ready
> -rw------- 1 postgres postgres 0 Apr 10 16:47 0000000100006C17000000D5.ready
> -rw------- 1 postgres postgres 0 Apr 10 16:47 0000000100006C17000000D6.ready
> -rw------- 1 postgres postgres 0 Apr 10 16:56 0000000100006C17000000D7.ready
> -rw------- 1 postgres postgres 0 Apr 10 16:56 0000000100006C17000000D8.ready
> -rw------- 1 postgres postgres 0 Apr 10 16:56 0000000100006C17000000D9.ready
> -rw------- 1 postgres postgres 0 Apr 10 16:56 0000000100006C17000000DA.ready
> -rw------- 1 postgres postgres 0 Apr 10 16:56 0000000100006C17000000DB.ready
> -rw------- 1 postgres postgres 0 Apr 10 16:56 0000000100006C17000000DC.ready
> -rw------- 1 postgres postgres 0 Apr 10 16:56 0000000100006C17000000DD.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000DE.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000DF.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000E0.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000E1.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000E2.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000E3.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000E4.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000E5.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000E6.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000E7.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000E8.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000E9.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000EA.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000EB.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000EC.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000ED.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000EE.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000EF.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000F0.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000F1.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000F2.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000F3.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000F4.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000F5.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000F6.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000F7.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000F8.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000F9.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000FA.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000FB.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000FC.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000FD.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000FE.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C17000000FF.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000000.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000001.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000002.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000003.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000004.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000005.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000006.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000007.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000008.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000009.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000000A.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000000B.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000000C.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000000D.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000000E.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000000F.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000010.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000011.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000012.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000013.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000014.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000015.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000016.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000017.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000018.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000019.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000001A.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000001B.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000001C.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000001D.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000001E.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000001F.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000020.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000021.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000022.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000023.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000024.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000025.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000026.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000027.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000028.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000029.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000002A.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000002B.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000002C.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000002D.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000002E.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000002F.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000030.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000031.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000032.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000033.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000034.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000035.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000036.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000037.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000038.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C1800000039.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000003A.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000003B.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000003C.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000003D.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:02 0000000100006C180000003E.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:07 0000000100006C180000003F.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:07 0000000100006C1800000040.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:07 0000000100006C1800000041.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:07 0000000100006C1800000042.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:07 0000000100006C1800000043.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:07 0000000100006C1800000044.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:07 0000000100006C1800000045.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:12 0000000100006C1800000046.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:12 0000000100006C1800000047.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:12 0000000100006C1800000048.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:12 0000000100006C1800000049.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:12 0000000100006C180000004A.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:12 0000000100006C180000004B.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:12 0000000100006C180000004C.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:12 0000000100006C180000004D.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:12 0000000100006C180000004E.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:12 0000000100006C180000004F.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:12 0000000100006C1800000050.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:12 0000000100006C1800000051.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:12 0000000100006C1800000052.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:12 0000000100006C1800000053.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:12 0000000100006C1800000054.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:12 0000000100006C1800000055.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C1800000056.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C1800000057.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C1800000058.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C1800000059.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C180000005A.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C180000005B.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C180000005C.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C180000005D.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C180000005E.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C180000005F.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C1800000060.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C1800000061.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C1800000062.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C1800000063.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C1800000064.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C1800000065.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C1800000066.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C1800000067.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C1800000068.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C1800000069.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C180000006A.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C180000006B.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C180000006C.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C180000006D.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C180000006E.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C180000006F.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C1800000070.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:17 0000000100006C1800000071.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:22 0000000100006C1800000072.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:22 0000000100006C1800000073.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:22 0000000100006C1800000074.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:22 0000000100006C1800000075.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:22 0000000100006C1800000076.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:22 0000000100006C1800000077.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:22 0000000100006C1800000078.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:22 0000000100006C1800000079.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:22 0000000100006C180000007A.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:22 0000000100006C180000007B.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:22 0000000100006C180000007C.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:22 0000000100006C180000007D.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:22 0000000100006C180000007E.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:22 0000000100006C180000007F.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:22 0000000100006C1800000080.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:22 0000000100006C1800000081.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:22 0000000100006C1800000082.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:27 0000000100006C1800000083.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:27 0000000100006C1800000084.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:27 0000000100006C1800000085.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:27 0000000100006C1800000086.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:27 0000000100006C1800000087.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:27 0000000100006C1800000088.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:27 0000000100006C1800000089.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:27 0000000100006C180000008A.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:27 0000000100006C180000008B.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:27 0000000100006C180000008C.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:27 0000000100006C180000008D.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:27 0000000100006C180000008E.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:27 0000000100006C180000008F.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:27 0000000100006C1800000090.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:27 0000000100006C1800000091.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:27 0000000100006C1800000092.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:27 0000000100006C1800000093.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:27 0000000100006C1800000094.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:33 0000000100006C1800000095.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:33 0000000100006C1800000096.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:33 0000000100006C1800000097.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:33 0000000100006C1800000098.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:33 0000000100006C1800000099.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:33 0000000100006C180000009A.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:33 0000000100006C180000009B.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:33 0000000100006C180000009C.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:33 0000000100006C180000009D.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:33 0000000100006C180000009E.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:33 0000000100006C180000009F.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:33 0000000100006C18000000A0.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:33 0000000100006C18000000A1.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:33 0000000100006C18000000A2.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:33 0000000100006C18000000A3.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:40 0000000100006C18000000A4.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:40 0000000100006C18000000A5.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:40 0000000100006C18000000A6.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:40 0000000100006C18000000A7.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:40 0000000100006C18000000A8.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:40 0000000100006C18000000A9.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:40 0000000100006C18000000AA.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:40 0000000100006C18000000AB.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:40 0000000100006C18000000AC.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000AD.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000AE.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000AF.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000B0.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000B1.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000B2.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000B3.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000B4.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000B5.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000B6.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000B7.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000B8.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000B9.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000BA.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000BB.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000BC.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000BD.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000BE.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000BF.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000C0.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000C1.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000C2.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000C3.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000C4.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000C5.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:49 0000000100006C18000000C6.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:55 0000000100006C18000000C7.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:55 0000000100006C18000000C8.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:55 0000000100006C18000000C9.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:55 0000000100006C18000000CA.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:07 0000000200006C1800000040.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:07 0000000200006C1800000041.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:07 0000000200006C1800000042.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:07 0000000200006C1800000043.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:07 0000000200006C1800000044.ready
> -rw------- 1 postgres postgres 0 Apr 10 17:07 0000000200006C1800000045.ready
> -rw------- 1 postgres postgres 0 Sep 16 12:27 0000000300007A120000002D.done
> -rw------- 1 postgres postgres 0 Sep 16 12:27 0000000300007A120000002E.done
> -rw------- 1 postgres postgres 0 Sep 16 12:28 0000000300007A120000002F.done
> -rw------- 1 postgres postgres 0 Sep 16 12:28 0000000300007A1200000030.done
> -rw------- 1 postgres postgres 0 Sep 16 12:28 0000000300007A1200000031.done
> -rw------- 1 postgres postgres 0 Sep 16 12:29 0000000300007A1200000032.done
> -rw------- 1 postgres postgres 0 Sep 16 12:29 0000000300007A1200000033.done
> -rw------- 1 postgres postgres 0 Sep 16 12:30 0000000300007A1200000034.done
> -rw------- 1 postgres postgres 0 Sep 16 12:30 0000000300007A1200000035.done
> -rw------- 1 postgres postgres 0 Sep 16 12:30 0000000300007A1200000036.done
> -rw------- 1 postgres postgres 0 Sep 16 12:31 0000000300007A1200000037.done
> -rw------- 1 postgres postgres 0 Sep 16 12:31 0000000300007A1200000038.done
> -rw------- 1 postgres postgres 0 Sep 16 12:31 0000000300007A1200000039.done
> -rw------- 1 postgres postgres 0 Sep 16 12:32 0000000300007A120000003A.done
> -rw------- 1 postgres postgres 0 Sep 16 12:32 0000000300007A120000003B.done
> -rw------- 1 postgres postgres 0 Sep 16 12:32 0000000300007A120000003C.done
> -rw------- 1 postgres postgres 0 Sep 16 12:33 0000000300007A120000003D.done
> -rw------- 1 postgres postgres 0 Sep 16 12:33 0000000300007A120000003E.done
> -rw------- 1 postgres postgres 0 Sep 16 12:33 0000000300007A120000003F.done
> -rw------- 1 postgres postgres 0 Sep 16 12:34 0000000300007A1200000040.done
> -rw------- 1 postgres postgres 0 Sep 16 12:34 0000000300007A1200000041.done
> -rw------- 1 postgres postgres 0 Sep 16 12:34 0000000300007A1200000042.done
> -rw------- 1 postgres postgres 0 Sep 16 12:35 0000000300007A1200000043.done
> -rw------- 1 postgres postgres 0 Sep 16 12:35 0000000300007A1200000044.done
> -rw------- 1 postgres postgres 0 Sep 16 12:36 0000000300007A1200000045.done
> -rw------- 1 postgres postgres 0 Sep 16 12:36 0000000300007A1200000046.done
> -rw------- 1 postgres postgres 0 Sep 16 12:36 0000000300007A1200000047.done
> -rw------- 1 postgres postgres 0 Sep 16 12:37 0000000300007A1200000048.done
> -rw------- 1 postgres postgres 0 Sep 16 12:37 0000000300007A1200000049.done
> -rw------- 1 postgres postgres 0 Sep 16 12:38 0000000300007A120000004A.done
> -rw------- 1 postgres postgres 0 Sep 16 12:38 0000000300007A120000004B.done
> -rw------- 1 postgres postgres 0 Sep 16 12:38 0000000300007A120000004C.done
> -rw------- 1 postgres postgres 0 Sep 16 12:38 0000000300007A120000004D.done
> -rw------- 1 postgres postgres 0 Sep 16 12:39 0000000300007A120000004E.done
> -rw------- 1 postgres postgres 0 Sep 16 12:39 0000000300007A120000004F.done
> -rw------- 1 postgres postgres 0 Sep 16 12:40 0000000300007A1200000050.done
> -rw------- 1 postgres postgres 0 Sep 16 12:40 0000000300007A1200000051.done
> -rw------- 1 postgres postgres 0 Sep 16 12:40 0000000300007A1200000052.done
> -rw------- 1 postgres postgres 0 Sep 16 12:41 0000000300007A1200000053.done
> -rw------- 1 postgres postgres 0 Sep 16 12:41 0000000300007A1200000054.done
> -rw------- 1 postgres postgres 0 Sep 16 12:42 0000000300007A1200000055.done
> -rw------- 1 postgres postgres 0 Sep 16 12:42 0000000300007A1200000056.done
> -rw------- 1 postgres postgres 0 Sep 16 12:42 0000000300007A1200000057.done
> -rw------- 1 postgres postgres 0 Sep 16 12:42 0000000300007A1200000058.done
> -rw------- 1 postgres postgres 0 Sep 16 12:43 0000000300007A1200000059.done
> -rw------- 1 postgres postgres 0 Sep 16 12:43 0000000300007A120000005A.done
> -rw------- 1 postgres postgres 0 Sep 16 12:43 0000000300007A120000005B.done
> -rw------- 1 postgres postgres 0 Sep 16 12:43 0000000300007A120000005C.done
> -rw------- 1 postgres postgres 0 Sep 16 12:44 0000000300007A120000005D.done
> -rw------- 1 postgres postgres 0 Sep 16 12:44 0000000300007A120000005E.done
> -rw------- 1 postgres postgres 0 Sep 16 12:44 0000000300007A120000005F.done
> -rw------- 1 postgres postgres 0 Sep 16 12:45 0000000300007A1200000060.done
> -rw------- 1 postgres postgres 0 Sep 16 12:45 0000000300007A1200000061.done
> -rw------- 1 postgres postgres 0 Sep 16 12:45 0000000300007A1200000062.done
> -rw------- 1 postgres postgres 0 Sep 16 12:46 0000000300007A1200000063.done
> -rw------- 1 postgres postgres 0 Apr 10 16:49 00000003.history.done
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000B5.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000B6.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000B7.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000B8.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000B9.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000BA.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000BB.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000BC.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000BD.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000BE.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000BF.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000C0.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000C1.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000C2.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000C3.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000C4.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000C5.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000C6.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000C7.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000C8.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000C9.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000CA.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000CB.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000CC.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000CD.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000CE.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000CF.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000D0.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000D1.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000D2.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000D3.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000D4.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000D5.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000D6.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000D7.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000D8.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000D9.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000DA.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000DB.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000DC.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000DD.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000DE.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000DF.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000E0.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000E1.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000E2.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000E3.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000E4.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000E5.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000E6.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000E7.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000E8.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000E9.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000EA.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000EB.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000EC.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000ED.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000EE.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000EF.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000F0.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000F1.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000F2.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000F3.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000F4.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000F5.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000F6.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000F7.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000F8.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000F9.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000FA.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000FB.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000FC.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000FD.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF4000000FE.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000000.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000001.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000002.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000003.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000004.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000005.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000006.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000007.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000008.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000009.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF50000000A.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF50000000B.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF50000000C.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF50000000D.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF50000000E.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF50000000F.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000010.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000011.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000012.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000013.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000014.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000015.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000016.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000017.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000018.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000019.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF50000001A.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF50000001B.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF50000001C.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF50000001D.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF50000001E.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF50000001F.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000020.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000021.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000022.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000023.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000024.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000025.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000026.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000027.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000028.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000029.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF50000002A.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF50000002B.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF50000002C.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF50000002D.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF50000002E.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF50000002F.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000030.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000031.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000032.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000033.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000034.ready
> -rw------- 1 postgres postgres 0 Apr 9 13:13 0000000800006BF500000035.ready
>
> I think most of them were generated to the moment right near the
> timeline switch, likely
> related to rebuilding of the master (and this replica) due to 9.3
> upgrade or for some
> other purposes. They do not correspond to the time when the database
> was experiencing
> issues described in the first post.
>
I suspect there is a problem while setting the archive_command that
resulting towards accumulating WAL *.ready* file in
pg_xlog/archive_status directory itself.
Please verify the archive_command that set at your end which should
resolve the issue. One example is provided for your reference.

Example:
archive_command = 'cp %p /tmp/PostgreSQL/archive/%f'
--

<http://www.shreeyansh.com>

Harshad Adalkonda
Database Administrator
harshad(dot)adalkonda(at)shreeyansh(dot)com <mailto:harshad(dot)adalkonda(at)shreeyansh(dot)com>

Office: +919552687400/8400
http://www.shreeyansh.com

Twitter <%20> Google Plus <%20> Youtube <%20> Linkedin
<http://in.linkedin.com/in/prashanthranjalkar> Instagram <%20> Pintrest
<%20> Dribbble <%20> Skype <skype:harsh_ur1>