Re: IRe: BUG #16792: silent corruption of GIN index resulting in SELECTs returning non-matching rows

On 20/07/2021 15:22, Pawel Kudzia wrote:
> On Tue, Jul 20, 2021 at 1:55 PM Heikki Linnakangas <hlinnaka(at)iki(dot)fi> wrote:
>> Pawel: Could you try the attached version of amcheck? Please apply the
>> attached patch (you'll have to un-apply the previous version or start
>> from a clean source tree), run "SELECT
>> gin_index_parent_check('entity_attribute_name_ids_gin');" again, and
>> send over the result? It should print a bunch of INFOs and WARNINGs to
>> the client's stderr, so a good way to capture that is something like this:
>>
>> psql -c "select
>> gin_index_parent_check('entity_attribute_name_ids_gin');" 2> amcheck-log
>>
>
> i'm attaching log of postgresql generated while running the command
> above and amcheck-log.
>
> thanks a lot!

Great! So it's reporting two kinds of issues. First this:

WARNING: index "entity_attribute_name_ids_gin": tid exceeds parent's
high key in postingTree leaf on block 343573

I looked at the amcheck code again, and that is in fact bogus; there is
nothing wrong with the high keys, but there is a memory management bug
in the amcheck patch that's causing it.

The other error it complains about looks like this:

...
INFO: key (1743225, 6) -> 363962
INFO: key (1746295, 15) -> 341281
INFO: key (1747712, 12) -> 343607
INFO: key (1749215, 4) -> 376056
INFO: key (262563, 1) -> 11547
INFO: key (3906513777, 43269) -> 31653888
INFO: key (536354808, 8196) -> 4432
ERROR: index "entity_attribute_name_ids_gin" has wrong tuple order in
posting tree, block 371806, offset 818

Looking at the amcheck code for that check, that's a false alarm too.
amcheck is calculating the size of this array incorrectly, thinking that
there are more entries on the page than there really are. The entries
above up to (1749215, 4) are valid, but the rest are not real entries,
just amcheck's misinterpretation. Unfortunately 'amcheck' stopped at
that error again.

Sigh. I fixed those issues in amcheck again. And also added some more
checks on the high keys. At least we're making the amcheck patch better,
even if we're not any wiser about the original issue...

Pawel: Can you test once again with the attached amcheck version?

- Heikki