"Tambet Matiisen" <tambet(dot)matiisen(at)gmail(dot)com> wrote:
> For a few days I've been getting this error from my nightly backup
> script:
>
> Warning: pg_dump: SQL command failed pg_dump: Error message from
> server: ERROR: found toasted toast chunk for toast value 260340218
> in pg_toast_260339342 pg_dump: The command was: COPY
> public.yhistud_urlcache (id, url, params, sess_id, content) TO
> stdout; pg_dumpall: pg_dump failed on database "yhistud", exiting
> Warning: Failed to dump pgsql cluster
So you don't have a current backup, and your database is corrupted.
(1) If you still have a backup from before you started getting
backup failures, keep it safe until everything has settled down and
is running well for several months.
(2) Stop PostgreSQL and do a full copy of the data directory and
everything under it to a backup medium or another machine. Keep
this copy safe for months, too.
> Yesterday I upgraded the server from 8.4.5 to 8.4.7, hoping that
> this error will go away, but no success.
Newer versions with more bug fixes may be less likely to contain
bugs which could cause corruption, but an upgrade like that is
unlikely to "heal" data which is already corrupted.
> I've been getting occasional errors from backup script for several
> months,
Do you know what those were?
> I have upgraded Linux kernel to 2.6.32, hoping that maybe the
> problem is in software RAID driver, but no changes, occasionally I
> still get errors.
Occasionally get what errors?
> I still have to do memory test on the server, but I doubt faulty
> memories are the problem, because otherwise the server behaves
> well.
So, no problems other than months of errors on backups? Never any
OS lockups, power losses, or other abrupt terminations of
operations?
Also, do you now or have you ever run the database with fsync = off
or full_page_writes = off?
It is very important to figure out how your data got corrupted;
otherwise you can't really trust this machine..
-Kevin