Re: BUG in postgres mathematic

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: "Robert B(dot) Easter" <reaster(at)comptechnews(dot)com>
Cc: Max Vaschenko <max(at)nino(dot)ru>, pgsql-bugs(at)postgresql(dot)org, Thomas Lockhart <lockhart(at)alumni(dot)caltech(dot)edu>
Subject: Re: BUG in postgres mathematic
Date: 2001-01-26 23:07:05
Message-ID: 12975.980550425@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

>> The fact that 5*27.81*100 != 27.81*100*5 is certainly a garden-variety
>> floating-point roundoff error. However, I think Max has a fair
>> complaint here: it seems float-to-int8 conversion is truncating, not
>> rounding like the other conversions to integer do.

I have changed float8-to-int8 to start with an rint() call, the same as
float8-to-int4 and float8-to-int2. This should give the same roundoff
behavior as the other cases, including round-to-nearest-even if your
hardware supports IEEE-compliant float math.

Curiously, this change exposed what I take to be a platform dependency
in the int8 regress test. It was computing
int8(float8(4567890123456789::int8)) and expecting to get back exactly
4567890123456789. However, that value is 53 bits long and so there is
no margin for error in a standard IEEE float8 value. I find that at
least on HP hardware, rint() treats the value as inexact and rounds to
nearest even:

regression=# select round(4567890123456788::float8) - 4567890123456780::float8;
?column?
----------
8
(1 row)

regression=# select round(4567890123456789::float8) - 4567890123456780::float8;
?column?
----------
8
(1 row)

regression=# select round(4567890123456790::float8) - 4567890123456780::float8;
?column?
----------
10
(1 row)

regression=#

Whether this is a bug in rint or spec-compliant behavior is unclear, but
I'll bet HP's hardware is not the only platform that behaves this way.
Since I'm not eager to try to develop a new set of platform-specific
int8 expected files at this late hour, I just diked out that test
instead...

regards, tom lane

In response to

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Barry Lind 2001-01-26 23:37:50 Re: [BUGS] no way in LargeObject API to detect short read?
Previous Message Tom Lane 2001-01-26 21:26:59 Re: select fails on indexed varchars.