Re: Solving hash table overrun problems

On Thu, Mar 03, 2005 at 17:05:40 -0500,
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
> * Estimate the number of batches N using the planner's estimate.
> We will always choose N a power of 2. A tuple's batch number is
> ((H div K) mod N).

If K is way low this could be very slow. Is there a way to do something
similar changing the hash function to H div KN? If you went down this
road you would probably want to use distinct primes for each new N.

> * Now begin scanning the outer join input. Tuples of batch number
> zero (according to the current calculation) can be matched to the
> current hashtable contents. Tuples of higher batch numbers are dropped
> into holding files for the outer input, one per batch.

For new keys at this step do you know their final disposition so that making
new hash entries won't be necessary?