Re: Improve search for missing parent downlinks in amcheck

On Sat, Apr 27, 2019 at 5:13 PM Alexander Korotkov
<a(dot)korotkov(at)postgrespro(dot)ru> wrote:
> Yes, increasing of Bloom filter size also helps. But my intention was
> to make non-lossy check here.

Why is that your intention? Do you want to do this as a feature for
Postgres 13, or do you want to treat this as a bug that we need to
backpatch a fix for?

Can we avoid the problem you saw with the Bloom filter approach by
using the real size of the index (i.e.
smgrnblocks()/RelationGetNumberOfBlocks()) to size the Bloom filter,
and/or by rethinking the work_mem cap? Maybe we can have a WARNING
that advertises that work_mem is probably too low?

The state->downlinkfilter Bloom filter should be small in almost all
cases, so I still don't fully understand your concern. With a 100GB
index, we'll have ~13 million blocks. We only need a Bloom filter that
is ~250MB to have less than a 1% chance of missing inconsistencies
even with such a large index. I admit that its unfriendly that users
are not warned about the shortage currently, but that is something we
can probably find a simple (backpatchable) fix for.

--
Peter Geoghegan