Re: BUG #14621: ERROR: compressed data is corrupt

Hi Andrew and Tom,

Thanks for your reply.

I tried the suggested function and it did show where the corrupted ctid is.

Thanks a lot for your help. Will try this function on all tables just to see how bad the corruption is.

Regards
Lara
-----Original Message-----
From: Andrew Gierth [mailto:andrew(at)tao11(dot)riddles(dot)org(dot)uk]
Sent: den 12 april 2017 15:01
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Lara Schembri <Lara(dot)Schembri(at)nyxgg(dot)com>; pgsql-bugs(at)postgresql(dot)org
Subject: Re: [BUGS] BUG #14621: ERROR: compressed data is corrupt

>>>>> "Tom" == Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:

>> Please note i have tried the chk function
>> declare t text;
>> begin t := $1;
>> return false;
>> exception when others then return true;
>> end;

>> which did not work.

Tom> You would get better responses if you defined what you meant by Tom> "did not work", but I'm going to guess that the issue is that this Tom> code failed to expose corrupted data. That's probably because it Tom> would have just assigned the bad datum to "t" without Tom> decompressing it.

The original intended use of that function (which is one I used to give out regularly to people on IRC when working with them on data corruption issues, and no doubt some of them have subsequently posted it on blogs or whatnot) is to pass in the whole-row var for $1 like so:

select ctid, id from brokentable t where chk(t);

the intent being to detect failures of external toast fetches or other corruption symptoms.

Unfortunately this is no longer as useful as it was, since a fix some time back now has chk(t) detoast the fields of t before entering the function, so the exception doesn't get caught. These days I usually have people use this one instead:

create function chk(tid) returns boolean language plpgsql as $f$
declare
r text;
begin
r := (select t from brokentable t where ctid=$1);
return false;
exception when others then
return true;
end;
$f$;

select ctid, id from brokentable t where chk(ctid);

In both versions of the function, it's relying on the fact that everything will get detoasted and decompressed as part of casting the record value to text.

--
Andrew (irc:RhodiumToad)