Re: Non-deterministic IndexTuple toast compression from index_form_tuple() + amcheck false positives

On Fri, Feb 1, 2019 at 6:27 PM Peter Geoghegan <pg(at)bowt(dot)ie> wrote:
> Attached draft patch fixes the bug by doing fairly simple
> normalization. I think that TOAST compression of datums in indexes is
> fairly rare in practice, so I'm not very worried about the fact that
> this won't perform as well as it could with indexes that have a lot of
> compressed datums. I think that the interface I've added might need to
> be expanded for other things in the future (e.g., to make amcheck work
> with nbtree-native duplicate compression), and not worrying about the
> performance too much helps with that goal.
>
> I'll pick this up next week, and likely commit a fix by Wednesday or
> Thursday if there are no objections. I'm not sure if the test case is
> worth including.

On second thought, the test should look like this:

$ psql --no-psqlrc --echo-queries -f bug_repro.sql
drop table if exists bug;
DROP TABLE
create table bug (buggy text);
CREATE TABLE
alter table bug alter column buggy set storage plain;
ALTER TABLE
create index toasty on bug(buggy);
CREATE INDEX
alter table bug alter column buggy set storage extended;
ALTER TABLE
insert into bug select repeat ('a', 2100);
INSERT 0 1
select bt_index_parent_check('toasty', true);
psql:bug_repro.sql:7: ERROR: heap tuple (0,1) from table "bug" lacks
matching index tuple within index "toasty"

This relies on the fact that the pg_attribute entry for the index is
more or less a straight copy of the corresponding pg_attribute entry
for the table at the time of the CREATE INDEX. I arrange to make
storage of the index attribute plain and storage for the heap
attribute extended. TOAST is applied inconsistently between
toast_insert_or_update() and index_form_tuple() without really relying
on implementation details that are subject to change.

--
Peter Geoghegan