From: | Дмитрий Шалашов <skaurus(at)gmail(dot)com> |
---|---|
To: | pgsql-admin(at)postgresql(dot)org |
Subject: | Make broken database start |
Date: | 2014-11-26 13:38:41 |
Message-ID: | CAKPeCUF8HGRyjxc-0ZoYEBg_16Ts1+LYyHLk0bHMS9vGMC0waw@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-admin |
Hi everyone,
(sorry if this is a dup - I'm not sure if my previous message get through)
I need an advice in database recovery. Postgresql 9.2.9.
Last night we moved one of our databases to another server following this
procedure: https://wiki.postgresql.org/wiki/Streaming_Replication
Replication, slave becomes master, pgbouncer restarts with new config,
everyone happy.
Until soon we noticed strange errors. Turns out data was broken somewhere -
we had unique indexes with duplicate records, we had missing chunks in
toast tables, and the like.
Scale of the problems wasn't clear and I fixed some of the most obvious
problems - recreated indices, cleared duplicates. Around 0:30 I started
database-wide VACUUM, it started printing messages about fixing things.
Everything seemed to work normally.
Around 3:30 database stopped responding to queries; it was noticed around
9:30 and I attempted to stop VACUUM (ctrl+c in psql command line). 30
minutes and nothing. I decided to restart postmaster. After another 30
minutes we switched back to old server. Already more than 6 hours passed
and new server postmaster is still shutting down. And I would like to get
it back to restore yesterday data.
At this point I could do with new server anything that will get database up
and running sooner. kill -9, start, wait for recovery? Server reboot?
Any help will be greatly appreciated.
Best regards,
Dmitriy Shalashov
From | Date | Subject | |
---|---|---|---|
Next Message | John R Pierce | 2014-11-26 17:20:20 | Re: Active/Active clustering in postgres |
Previous Message | Jan Lentfer | 2014-11-26 11:00:22 | Re: Active/Active clustering in postgres |