Re: init_htab causes SIGFPE (or worse) due to miscalculation for large nbuckets

Jeff Davis <pgsql(at)j-davis(dot)com> writes:
> If nbuckets is around 2^30+1, which can happen if work_mem is very high,
> then:
> nbuckets = 1 << my_log2(lnbuckets);
> ends up as 1 << 31, which is negative, leading to a SIGFPE on my
> machine, but I think it can also lead to an infinite loop or a crash
> (after corrupting the HASHHDR).
> ...
> I think we should just cap nbuckets at 1 << 30 in init_htab.

Yeah. After looking at other uses of my_log2, it seems like
hash_estimate_size and hash_select_dirsize probably should also
bound their inputs to avoid overflow of 1 << my_log2() calculations.

Alternatively, maybe we should hack my_log2 to do that bounding.
As-is, it seems like trouble waiting to happen. This won't protect
init_htab, which needs the shift result to fit in int not long.
But my_log2 is just plain broken for inputs larger than LONG_MAX/2,
anyway.

regards, tom lane