Re: Negative value of numGroups

On 5/17/22 20:34, Tom Lane wrote:
> Andrey Lepikhov <a(dot)lepikhov(at)postgrespro(dot)ru> writes:
>> On 13/5/2022 11:56, Andrey Lepikhov wrote:
>>> (long) Min(numGroups, (double) LONG_MAX);
>>>
>>> can return negative value, if numGroups > LONG_MAX.
>
> I see your point, but I don't think that repeating the same finicky
> and undocumented coding pattern in multiple places is a future-proof
> fix. I'm inclined to think we should invent a function along the
> lines of "long clamp_double_to_long(double x)". Given where it's
> used, maybe putting it beside clamp_row_est() would be good.
>
> regards, tom lane
Next version of the patch.
As I see, now we trying to use Cardinality type instead of double.
So, I named casting routine as clamp_cardinality_to_long.

--
Regards
Andrey Lepikhov
Postgres Professional