Re: amcheck (B-Tree integrity checking tool)

On Mon, Oct 17, 2016 at 10:46 AM, Noah Misch <noah(at)leadboat(dot)com> wrote:
> - Detect impossible conditions in the hint bits. A tuple should not have both
> HEAP_XMAX_COMMITTED and HEAP_XMAX_INVALID. Every tuple bearing
> HEAP_ONLY_TUPLE should bear HEAP_UPDATED. HEAP_HASVARWIDTH should be true
> if and only if the tuple has a non-NULL value in a negative-typlen column,
> possibly a dropped column. A tuple should not have both HEAP_KEYS_UPDATED
> and HEAP_XMAX_LOCK_ONLY.
>
> - Report evidence of upgrades from older versions. If the tool sees
> HEAP_MOVED_IN or HEAP_MOVED_OFF, it can report that the cluster was
> binary-upgraded from 8.3 or 8.4. If the user did not upgrade from such a
> version, the user should assume corruption.
>
> - Check VARSIZE() of each variable-length datum. Corrupt lengths might direct
> you to seek past the end of the tuple, or they might imply excess free space
> at the end of the tuple.
>
> - Verify agreement between CLOG, MULTIXACT, and hint bits. If the hint bits
> include HEAP_XMAX_LOCK_ONLY, the multixact should not contain a
> MultiXactStatusUpdate member. README.tuplock documents other invariants.
> If the tool sees a tuple passing HEAP_LOCKED_UPGRADED, it can report that
> the cluster was binary-upgraded from a version in [8.3, 9.1].
>
> - Verify that TOAST pointers (in non-dropped columns of visible tuples) point
> to valid data in the TOAST relation. This is much more expensive than the
> other checks I've named, so it should be optional.

There is a lot of value in that actually. I got bitten lately by an
issue where this got corrupted because of an incorrect failover flow.

> - If VM_ALL_VISIBLE() or VM_ALL_FROZEN() passes for a particular page, verify
> that the visibility data stored in the page is compatible with that claim.
>
> - Examine PageHeaderData values. If pd_checksum is non-zero in a cluster with
> checksums disabled, the cluster was binary-upgraded from [8.3, 9.2].

Agreed basically on all those items, though the first version does not
need to do everything :)

Honestly just btree checks have enough value to validate the
integration of this module. You can get that as "I'll look at your
patch and provide a review soon because I have a hell lot of cases
where this will be useful".
--
Michael