Re: Floating point error

From: John R Pierce <pierce(at)hogranch(dot)com>
To: pgsql-general(at)postgresql(dot)org
Subject: Re: Floating point error
Date: 2013-02-25 02:20:11
Message-ID: 512ACA5B.6040506@hogranch.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

On 2/24/2013 6:13 PM, Tom Duffey wrote:
> - The Java app on production shows "10.3884573" while the test app shows "10.3885"

'real' is single precision, which is only about 6 digits of decimal
accuracy. if your java variables were double precision, you probably
should have used double in postgres too.

0.1 decimal is a repeating fraction in binary.

--
john r pierce 37N 122W
somewhere on the middle of the left coast

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message John R Pierce 2013-02-25 02:26:33 Re: Floating point error
Previous Message Tom Duffey 2013-02-25 02:13:20 Floating point error

Browse pgsql-hackers by date

  From Date Subject
Next Message John R Pierce 2013-02-25 02:26:33 Re: Floating point error
Previous Message Tom Duffey 2013-02-25 02:13:20 Floating point error