Significant Digits in Floating Point Datatype

From: Lee Hachadoorian <lee(dot)hachadoorian(at)gmail(dot)com>
To: "pgsql-general(at)postgresql(dot)org" <pgsql-general(at)postgresql(dot)org>
Subject: Significant Digits in Floating Point Datatype
Date: 2011-11-20 18:29:37
Message-ID: 4EC94711.4000007@gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

I would like to store some in a single array some data that is
conceptually related, but some of the data is floating point, and some
of it is integer. Obviously the integer data *can* be stored as double
precision, but I need to know about potential loss of precision. Double
precision has "a precision of at least 15 digits." I would assume that
that would mean that for double precision, 15 digits of data would be
faithfully preserved. But I found a question on the list where a value
stored as 955.60 sometimes returns as 955.599999999998.
(http://archives.postgresql.org/pgsql-general/2011-08/msg00144.php) If
this is the case, what does "a precision of at least [x] digits"
actually mean? And can I reliably retrieve the original integer by
casting to int (or bigint) if the number of digits in the original
integer is less than 15?

Regards,
--Lee

--
Lee Hachadoorian
PhD, Earth& Environmental Sciences (Geography)
Research Associate, CUNY Center for Urban Research
http://freecity.commons.gc.cuny.edu

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Scott Marlowe 2011-11-20 18:56:23 Re: Installed. Now what?
Previous Message Raghavendra 2011-11-20 18:13:26 How to install pgfincore with PG 9.1