Re: [HACKERS] Hashjoin status report

Tom Lane wrote:
>
> I've committed fixes that deal with all of the coredump problems
> I could find in nodeHash.c (there were several :-().
>
> But the code still has a fundamental design flaw: it uses a fixed-size
> overflow area to hold tuples that don't fit into the hashbuckets they
> are assigned to. This means you get "hashtable out of memory" errors
> if the distribution of tuples is skewed enough, or if the number of
> hashbuckets is too small because the system underestimated the number
> of tuples in the relation. Better than a coredump I suppose, but still
> very bad, especially since the optimizer likes to use hashjoins more
> than it used to.
>
> What I would like to do to fix this is to store the tuples in a Portal
> instead of in a fixed-size palloc block. While at it, I'd rip out the
> "relative vs. absolute address" cruft that is in the code now.
> (Apparently there was once some thought of using a shared memory block
> so that multiple processes could share the work of a hashjoin. All that
> remains is ugly, bug-prone code ...)

Fix it! Testing is easy...
Though, I would use chain of big blocks for overflow area,
not Portal - it's too big thing to be directly used in join method.

Vadim