| From: | Stephan Szabo <sszabo(at)megazone(dot)bigpanda(dot)com> |
|---|---|
| To: | Iain <iain(at)mst(dot)co(dot)jp> |
| Cc: | Adam Lancaster <alancaster(at)preventsys(dot)com>, pgsql-sql(at)postgresql(dot)org |
| Subject: | Re: Possible rounding error of large float values? |
| Date: | 2004-08-25 01:41:37 |
| Message-ID: | 20040824183612.M67501@megazone.bigpanda.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-sql |
On Wed, 25 Aug 2004, Iain wrote:
>
> # select 9223372036854775807 = 9223372036854775807::float;
> ?column?
> ----------
> t
> (1 row)
> This and the fact that it's still possible to find the row using the
> original value would seem to indicate that the rounding is just a display
> artifact..
I don't think that's true since the number ending in 807 is equal to a
version ending in 808.
sszabo=# select 9223372036854775807 = 9223372036854775808::float;
?column?
----------
t
(1 row)
(In this next one I'm casting both because I want to choose the types of
both literals)
sszabo=# select 9223372036854775807::float =
9223372036854775808::numeric;
?column?
----------
t
(1 row)
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Iain | 2004-08-25 02:48:11 | Re: Possible rounding error of large float values? |
| Previous Message | Iain | 2004-08-25 01:32:29 | Re: Possible rounding error of large float values? |