Re: Replication terminated due to PANIC

If its really index corruption, then you should be able to fix it by
reindexing. However, that doesn't explain what caused the corruption.
Perhaps your hardware is bad in some way?

On Wed, Apr 24, 2013 at 10:46 PM, Adarsh Sharma <eddy(dot)adarsh(at)gmail(dot)com> wrote:
> Thanks Sergey for such a quick response, but i dont think this is some patch
> problem because we have other DB servers also running fine on same version
> and message is also different :
>
> host= PANIC: _bt_restore_page: cannot add item to page
>
> And the whole day replication is working fine but at midnight when log
> rotates it shows belows msg :
>
> 2013-04-24 00:00:00 UTC [26989]: [4945032-1] user= db= host= LOG:
> checkpoint starting: time
> 2013-04-24 00:00:00 UTC [26989]: [4945033-1] user= db= host= ERROR: could
> not open file "global/14078": No such file or directory
>
> 2013-04-24 00:00:00 UTC [26989]: [4945034-1] user= db= host= CONTEXT:
> writing block 0 of relation global/14078
> 2013-04-24 00:00:00 UTC [26989]: [4945035-1] user= db= host= WARNING: could
> not write block 0 of global/14078
>
> 2013-04-24 00:00:00 UTC [26989]: [4945036-1] user= db= host= DETAIL:
> Multiple failures --- write error might be permanent.
>
> Looks like some index corruption.
>
>
> Thanks
>
>
>
>
>
>
> On Thu, Apr 25, 2013 at 8:14 AM, Sergey Konoplev <gray(dot)ru(at)gmail(dot)com> wrote:
>>
>> On Wed, Apr 24, 2013 at 5:05 PM, Adarsh Sharma <eddy(dot)adarsh(at)gmail(dot)com>
>> wrote:
>> > I have a Postgresql 9.2 instance running on a CentOS6.3 box.Yesterday i
>> > setup a hot standby by using pgbasebackup. Today i got the below alert
>> > from
>> > standby box :
>> >
>> > [1] (from line 412,723)
>> > 2013-04-24 23:07:18 UTC [13445]: [6-1] user= db= host= PANIC:
>> > _bt_restore_page: cannot add item to page
>> >
>> > When i check, the replication is terminated due to slave DB shutdown.
>> > From
>> > the logs i can see below messages :-
>>
>> I am not sure that it is your situation but take a look at this thread:
>>
>>
>> http://www.postgresql.org/message-id/CAL_0b1t=WuM6roO8dki=w8DhH8P8whhohbPjReymmQUrOcNT2A@mail.gmail.com
>>
>> There is a patch by Andres Freund in the end of the discussion. Three
>> weeks have passed after I installed the patched version and it looks
>> like the patch fixed my issue.
>>
>> >
>> > 2013-04-24 23:17:16 UTC [26989]: [5360083-1] user= db= host= ERROR:
>> > could
>> > not open file "global/14078": No such file or directory
>> > 2013-04-24 23:17:16 UTC [26989]: [5360084-1] user= db= host= CONTEXT:
>> > writing block 0 of relation global/14078
>> > 2013-04-24 23:17:16 UTC [26989]: [5360085-1] user= db= host= WARNING:
>> > could
>> > not write block 0 of global/14078
>> > 2013-04-24 23:17:16 UTC [26989]: [5360086-1] user= db= host= DETAIL:
>> > Multiple failures --- write error might be permanent.
>> >
>> > I checked in global directory of master, the directory 14078 doesn't
>> > exist.
>> >
>> > Anyone has faced above issue ?
>> >
>> > Thanks
>>
>>
>>
>> --
>> Kind regards,
>> Sergey Konoplev
>> Database and Software Consultant
>>
>> Profile: http://www.linkedin.com/in/grayhemp
>> Phone: USA +1 (415) 867-9984, Russia +7 (901) 903-0499, +7 (988) 888-1979
>> Skype: gray-hemp
>> Jabber: gray(dot)ru(at)gmail(dot)com
>
>

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
L. Friedman netllama(at)gmail(dot)com
LlamaLand https://netllama.linux-sxs.org