Re: BUG #15519: Casting float4 into int4 gets the wrong sign instead of "integer out of range" error

From: Andrew Gierth <andrew(at)tao11(dot)riddles(dot)org(dot)uk>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: victor(at)magic(dot)io, pgsql-bugs(at)lists(dot)postgresql(dot)org
Subject: Re: BUG #15519: Casting float4 into int4 gets the wrong sign instead of "integer out of range" error
Date: 2018-11-23 23:42:50
Message-ID: 87zhtzqs9o.fsf@news-spur.riddles.org.uk
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

>>>>> "Tom" == Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:

Tom> /*
Tom> * Range check. We must be careful here that the boundary values are
Tom> * expressed exactly in the appropriate float domain; we assume float4
Tom> * is going to round off INT_MAX to a power of 2. Also note assumption
Tom> * that rint() will pass through a NaN or Inf unchanged.
Tom> */
Tom> if (unlikely(num < (float4) INT_MIN || num >= (float4) INT_MAX || isnan(num)))
Tom> ereport(...);

Looking around, another approach I've seen (which I like better) is to
use

if (num < (float4)INT_MIN || num >= -(float4)INT_MIN || ...

which doesn't rely on assumptions about how the compiler is going to
round INT_MAX. (It does rely on two's complement representation of ints,
but that assumption is known to be safe.)

--
Andrew (irc:RhodiumToad)

In response to

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Tom Lane 2018-11-23 23:52:32 Re: BUG #15519: Casting float4 into int4 gets the wrong sign instead of "integer out of range" error
Previous Message Tom Lane 2018-11-23 23:03:47 Re: BUG #15519: Casting float4 into int4 gets the wrong sign instead of "integer out of range" error