Re: [HACKERS] Hashjoin status report

On Thu, 6 May 1999, Tom Lane wrote:

> I've committed fixes that deal with all of the coredump problems
> I could find in nodeHash.c (there were several :-().
>
> But the code still has a fundamental design flaw: it uses a fixed-size
> overflow area to hold tuples that don't fit into the hashbuckets they
> are assigned to. This means you get "hashtable out of memory" errors
> if the distribution of tuples is skewed enough, or if the number of
> hashbuckets is too small because the system underestimated the number
> of tuples in the relation. Better than a coredump I suppose, but still
> very bad, especially since the optimizer likes to use hashjoins more
> than it used to.
>
> What I would like to do to fix this is to store the tuples in a Portal
> instead of in a fixed-size palloc block. While at it, I'd rip out the
> "relative vs. absolute address" cruft that is in the code now.
> (Apparently there was once some thought of using a shared memory block
> so that multiple processes could share the work of a hashjoin. All that
> remains is ugly, bug-prone code ...)
>
> The reason I bring all this up is that it'd be a nontrivial revision
> to nodeHash.c, and I'm uncomfortable with the notion of making such a
> change this late in a beta cycle. On the other hand it *is* a bug fix,
> and a fairly important one IMHO.
>
> Opinions? Should I plow ahead, or leave this to fix after 6.5 release?

Estimate of time involved to fix this? vs likelihood of someone
triggering the bug in production?

Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy(at)hub(dot)org secondary: scrappy(at){freebsd|postgresql}.org