corrupted item pointer in streaming based replication

From: Jigar Shah <jshah(at)pandora(dot)com>
To: "pgsql-general(at)postgresql(dot)org" <pgsql-general(at)postgresql(dot)org>
Subject: corrupted item pointer in streaming based replication
Date: 2013-04-01 22:45:27
Message-ID: 1E737D138B89104D8A7853F7DD23177DB5F601@SF1-EXMBX-2.ad.savagebeast.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi,

Few days ago we started getting the below message and postgres on our server(streaming replication secondary) would not startup. I am wondering what are our options at this point. Can we do something to fix this?

2013-03-27 11:00:47.281 PDT LOG: recovery restart point at 161A/17108AA8
2013-03-27 11:00:47.281 PDT DETAIL: last completed transaction was at log time 2013-03-27 11:00:47.241236-07
2013-03-27 11:00:47.520 PDT LOG: restartpoint starting: xlog

2013-03-27 11:07:51.348 PDT FATAL: corrupted item pointer: offset = 0, size = 0
2013-03-27 11:07:51.348 PDT CONTEXT: xlog redo split_l: rel 1663/16384/115085 left 4256959, right 5861610, next 5044459, level 0, firstright 192
2013-03-27 11:07:51.716 PDT LOG: startup process (PID 5959) exited with exit code 1
2013-03-27 11:07:51.716 PDT LOG: terminating any other active server processes

Before the secondaries crashed we also noticed the below error message in postgres logs on the primary few days ago.

[d: u:postgres p:2498 7] ERROR: could not access status of transaction 837550133
DETAIL: Could not open file "pg_clog/031E": No such file or directory. [u:postgres p:2498 9]

[d: u:radio p:31917 242] ERROR: could not open file "base/16384/114846.39" (target block 360448000): No such file or directory [d: u:radio p:31917 243]

I'd like to know what can we do to fix this situation.

Thanks
Jigar

Browse pgsql-general by date

  From Date Subject
Next Message Julian 2013-04-02 02:12:49 Re: Using varchar primary keys.
Previous Message Juan Pablo Cook 2013-04-01 21:19:41 Re: JDBC Array double precision [] error