Re: Postgresql + corrupted disk = data loss. (Need help for database recover)

Hello!

i've don't try to do reindex. There was enough space.

And i have a full data-directory backup, when i've stop server, before
start.

2011/12/2 Venkat Balaji <venkat(dot)balaji(at)verse(dot)in>

>
> 2011/12/2 Oleg Serov <serovov(at)gmail(dot)com>
>
>> And, i'm an idiot.
>>
>> My DB version:
>> PostgreSQL 8.4.9 on x86_64-redhat-linux-gnu, compiled by GCC gcc (GCC)
>> 4.1.2 20080704 (Red Hat 4.1.2-51), 64-bit
>>
>>
>>
>> 2011/12/2 Oleg Serov <serovov(at)gmail(dot)com>
>>
>>> Hello, i have a problem.
>>>
>>> I've got a production server, working fine. Then i've got strange error:
>>> > ERROR: right sibling's left-link doesn't match: block 147 links to
>>> 407 instead of expected 146 in index "order_status_key"'
>>> And decidet to backup all server. So i shut-down VPS with server and
>>> backup all data.
>>> Then, after i booted it - and then - i've got Data loss.
>>>
>>
> This seems to be an Index corruption. Did you try re-indexing ? Index
> creation might have failed, re-indexing would re-organize the Index tuples.
> If you are sure about disk corruption, try and "re-create" or "create
> concurrent Index" on a different disk.
>

>
>
>> I've lost data, that have been written to DB around 10-100 hours
>>> (different tables, have different last updated value).
>>>
>>> Then i've analyzed log, and found this:
>>> 7 days ago appears this errors:
>>> db= LOG: could not rename temporary statistics file
>>> "pg_stat_tmp/pgstat.tmp" to "pg_stat_tmp/pgstat.stat":
>>> db= WARNING: pgstat wait timeout
>>> ERROR: missing chunk number 0 for toast value 2550017 in pg_toast_17076
>>
>>
> This should be a free space issue, do you have enough space in
> "pg_stat_tmp" disk ?
>

>
> 5 days ago:
>>> a lot of: ERROR: xlog flush request F/DC1A22D8 is not satisfied ---
>>> flushed only to F/526512E0
>>> 83238 db= WARNING: could not write block 54 of base/16384/2619
>>> 83239 db= CONTEXT: writing block 54 of relation base/16384/2619
>>>
>> And today:
>>> 18 db= LOG: could not open file "pg_xlog/000000010000000F00000052"
>>> (log file 15, segment 82):
>>> 19 db= ERROR: xlog flush request F/DC1A22D8 is not satisfied ---
>>> flushed only to F/52FDF0E0
>>>
>>
>>> There is any ability to recover fresh data from database?
>>>
>>
> What kind of backups you have available ?
>

>
>
> Thanks
> VB
>

--
С уважением

Олег