Re: Adjusting hash join memory limit to handle batch explosion

Hi,

Here's a slightly simplified version of the "balancing" patch. I decided
to stop increasing the nbucket value at runtime, even if the hashtable
grows larger than the memory limit (which is what we used to calculate
the initial nbucket value in ExecChooseHashTableSize).

Without the nbucket changes the patch does not need to change how
ExecHashGetBucketAndBatch() calculates bucketno/batchno, which added
quite a bit of complexity (the patch from a couple days ago simply
inverted bits in the hash when calculating batchno, but I'm sure there
are other ways to do this).

This doesn't mean we can't adjust nbucket at all - we just can't do that
at runtime, after we started building the hash. We still can adjust
nbucket in ExecChooseHashTableSize(), if we know we'll need many batches
at that point.

FWIW this is not a new issue, introduced by this patch. We can already
have issues with nbucket being too low if we disable growth of the hash
table early, in which case it can get almost arbitrarily large. And we
don't resize nbucket in that case either. Granted, we usually disable
growth in cases when there are duplicate values/hashes, so increasing
nbucket would not really do much (the tuples would still go to the same
bucket). But there are ways to disable disable growth early.

The 0002 patch helps with those cases a bit by retrying the nbatch
growth after a while, instead of disabling it forever.

0003 is just a WIP patch with a couple reproducers for the issues.

regards

--
Tomas Vondra