Re: After replication failover: could not read block X in file Y read only 0 of 8192 bytes

On Mon, May 30, 2016 at 11:37 PM, Brian Sutherland <brian(at)vanguardistas(dot)net>
wrote:

> I'm running a streaming replication setup with PostgreSQL 9.5.2 and have
> started seeing these errors on a few INSERTs:
>
> ERROR: could not read block 8 in file "base/3884037/3885279": read
> only 0 of 8192 bytes
>

These errors are occurring on master or slave ?

> on a few tables. If I look at that specific file, it's only 6 blocks
> long:
>
> # ls -la base/3884037/3885279
> -rw------- 1 postgres postgres 49152 May 30 12:56 base/3884037/3885279
>
> It seems that this is the case on most tables in this state. I havn't
> seen any error on SELECT and I can SELECT * on the all tables I know
> have this problem. The database is machine is under reasonable load.
>

So, the filenodes generating this error belong to a Table ? or an Index ?

> On some tables an "ANALYZE tablename" causes the error.
>
> We recently had a streaming replication failover after loading a large
> amount of data with pg_restore. The problems seem to have started after
> that, but I'm not perfectly sure.
>

pg_restore has completed successfully ? When pg_restore was running, did
you see anything suspicious in the postgresql logfiles ?

I have data_checksums switched on so am suspecting a streaming
> replication bug. Anyone know of a recent bug which could have caused
> this?
>

I cannot conclude at this point. I encountered these kind of errors with
Indexes and re-indexing fixed them.

Regards,
Venkata B N

Fujitsu Australia