From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
---|---|
To: | "Robert B(dot) Easter" <reaster(at)comptechnews(dot)com> |
Cc: | Max Vaschenko <max(at)nino(dot)ru>, pgsql-bugs(at)postgresql(dot)org, Thomas Lockhart <lockhart(at)alumni(dot)caltech(dot)edu> |
Subject: | Re: BUG in postgres mathematic |
Date: | 2001-01-26 23:07:05 |
Message-ID: | 12975.980550425@sss.pgh.pa.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-bugs |
>> The fact that 5*27.81*100 != 27.81*100*5 is certainly a garden-variety
>> floating-point roundoff error. However, I think Max has a fair
>> complaint here: it seems float-to-int8 conversion is truncating, not
>> rounding like the other conversions to integer do.
I have changed float8-to-int8 to start with an rint() call, the same as
float8-to-int4 and float8-to-int2. This should give the same roundoff
behavior as the other cases, including round-to-nearest-even if your
hardware supports IEEE-compliant float math.
Curiously, this change exposed what I take to be a platform dependency
in the int8 regress test. It was computing
int8(float8(4567890123456789::int8)) and expecting to get back exactly
4567890123456789. However, that value is 53 bits long and so there is
no margin for error in a standard IEEE float8 value. I find that at
least on HP hardware, rint() treats the value as inexact and rounds to
nearest even:
regression=# select round(4567890123456788::float8) - 4567890123456780::float8;
?column?
----------
8
(1 row)
regression=# select round(4567890123456789::float8) - 4567890123456780::float8;
?column?
----------
8
(1 row)
regression=# select round(4567890123456790::float8) - 4567890123456780::float8;
?column?
----------
10
(1 row)
regression=#
Whether this is a bug in rint or spec-compliant behavior is unclear, but
I'll bet HP's hardware is not the only platform that behaves this way.
Since I'm not eager to try to develop a new set of platform-specific
int8 expected files at this late hour, I just diked out that test
instead...
regards, tom lane
From | Date | Subject | |
---|---|---|---|
Next Message | Barry Lind | 2001-01-26 23:37:50 | Re: [BUGS] no way in LargeObject API to detect short read? |
Previous Message | Tom Lane | 2001-01-26 21:26:59 | Re: select fails on indexed varchars. |