Re: Precision errors in float8 type casting (as of 7.3.2 and

From: Stephan Szabo <sszabo(at)megazone23(dot)bigpanda(dot)com>
To: Philip Edelbrock <phil(at)edgedesign(dot)us>
Cc: <pgsql-bugs(at)postgresql(dot)org>
Subject: Re: Precision errors in float8 type casting (as of 7.3.2 and
Date: 2003-07-10 04:44:02
Message-ID: 20030709213211.R28062-100000@megazone23.bigpanda.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs


On Wed, 9 Jul 2003, Philip Edelbrock wrote:

> phil=# select (3.85::float4)::float8;
> float8
> ------------------
> 3.84999990463257
> (1 row)
>
> (Or substitute 3.85 for any number with something other than 0 to the
> right of the decimal point, or pull the same values from any table which
> stores in float4/real format.)
>
> Obviously, this is wrong and should return 3.85. We traced this down on

I see no obviously about it. Once you've placed a value in a float you
are accepting the chance of some precision loss. When we print a float4
we can print it with an amount of precision that generally limits this
(although you'll see things like 3.849998 -> 3.5), but once you cast it to
a float8 those values are distinguishably different. In theory one could
keep the history of the value around to determine a precision, but that
doesn't really seem better in general.

In response to

Browse pgsql-bugs by date

  From Date Subject
Next Message Stephan Szabo 2003-07-10 04:45:57 Re: Out of memory
Previous Message Philip Edelbrock 2003-07-10 00:16:08 Precision errors in float8 type casting (as of 7.3.2 and some earlier releases at least)