Re: Possible rounding error of large float values?

From: Josh Berkus <josh(at)agliodbs(dot)com>
To: "Adam Lancaster" <alancaster(at)preventsys(dot)com>, <pgsql-sql(at)postgresql(dot)org>
Subject: Re: Possible rounding error of large float values?
Date: 2004-08-24 19:52:30
Message-ID: 200408241252.30182.josh@agliodbs.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-sql

Adam,

> 9223372036854775807
>
> It gets selected out as:
>
> 9.22337203685478E18

This is a property of FLOAT data types. They round.

> Which appears to be rounded. When we cast it to numeric type we get:
>
> 9223372036854780000
>
> Which also is rounded. It is still possible to find the row using the
> original value

Hmmm ... is 15 digits the limit of NUMERIC? It may be.

> Is this expected behavior?

Yes.

--
Josh Berkus
Aglio Database Solutions
San Francisco

In response to

Responses

Browse pgsql-sql by date

  From Date Subject
Next Message Oliver Elphick 2004-08-24 20:21:15 Re: Possible rounding error of large float values?
Previous Message Bruno Mueller 2004-08-24 15:10:11 refer to computed columns