Re: Replication terminated due to PANIC

Thanks Sergey for such a quick response, but i dont think this is some
patch problem because we have other DB servers also running fine on same
version and message is also different :

host= PANIC: _bt_restore_page: cannot add item to page

And the whole day replication is working fine but at midnight when log
rotates it shows belows msg :

2013-04-24 00:00:00 UTC [26989]: [4945032-1] user= db= host= LOG:
checkpoint starting: time
2013-04-24 00:00:00 UTC [26989]: [4945033-1] user= db= host= ERROR:
could not open file "global/14078": No such file or directory
2013-04-24 00:00:00 UTC [26989]: [4945034-1] user= db= host= CONTEXT:
writing block 0 of relation global/14078
2013-04-24 00:00:00 UTC [26989]: [4945035-1] user= db= host= WARNING:
could not write block 0 of global/14078
2013-04-24 00:00:00 UTC [26989]: [4945036-1] user= db= host= DETAIL:
Multiple failures --- write error might be permanent.

Looks like some index corruption.

Thanks

On Thu, Apr 25, 2013 at 8:14 AM, Sergey Konoplev <gray(dot)ru(at)gmail(dot)com> wrote:

> On Wed, Apr 24, 2013 at 5:05 PM, Adarsh Sharma <eddy(dot)adarsh(at)gmail(dot)com>
> wrote:
> > I have a Postgresql 9.2 instance running on a CentOS6.3 box.Yesterday i
> > setup a hot standby by using pgbasebackup. Today i got the below alert
> from
> > standby box :
> >
> > [1] (from line 412,723)
> > 2013-04-24 23:07:18 UTC [13445]: [6-1] user= db= host= PANIC:
> > _bt_restore_page: cannot add item to page
> >
> > When i check, the replication is terminated due to slave DB shutdown.
> From
> > the logs i can see below messages :-
>
> I am not sure that it is your situation but take a look at this thread:
>
>
> http://www.postgresql.org/message-id/CAL_0b1t=WuM6roO8dki=w8DhH8P8whhohbPjReymmQUrOcNT2A@mail.gmail.com
>
> There is a patch by Andres Freund in the end of the discussion. Three
> weeks have passed after I installed the patched version and it looks
> like the patch fixed my issue.
>
> >
> > 2013-04-24 23:17:16 UTC [26989]: [5360083-1] user= db= host= ERROR:
> could
> > not open file "global/14078": No such file or directory
> > 2013-04-24 23:17:16 UTC [26989]: [5360084-1] user= db= host= CONTEXT:
> > writing block 0 of relation global/14078
> > 2013-04-24 23:17:16 UTC [26989]: [5360085-1] user= db= host= WARNING:
> could
> > not write block 0 of global/14078
> > 2013-04-24 23:17:16 UTC [26989]: [5360086-1] user= db= host= DETAIL:
> > Multiple failures --- write error might be permanent.
> >
> > I checked in global directory of master, the directory 14078 doesn't
> exist.
> >
> > Anyone has faced above issue ?
> >
> > Thanks
>
>
>
> --
> Kind regards,
> Sergey Konoplev
> Database and Software Consultant
>
> Profile: http://www.linkedin.com/in/grayhemp
> Phone: USA +1 (415) 867-9984, Russia +7 (901) 903-0499, +7 (988) 888-1979
> Skype: gray-hemp
> Jabber: gray(dot)ru(at)gmail(dot)com
>