Re: Floating point error

From: Edson Richter <edsonrichter(at)hotmail(dot)com>
To: pgsql-general(at)postgresql(dot)org
Subject: Re: Floating point error
Date: 2013-02-25 02:40:01
Message-ID: BLU0-SMTP278DB86DF2833DB30D46C06CFF30@phx.gbl
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

Em 24/02/2013 23:26, John R Pierce escreveu:
> On 2/24/2013 6:20 PM, John R Pierce wrote:
>> On 2/24/2013 6:13 PM, Tom Duffey wrote:
>>> - The Java app on production shows "10.3884573" while the test app
>>> shows "10.3885"
>>
>> 'real' is single precision, which is only about 6 digits of decimal
>> accuracy. if your java variables were double precision, you
>> probably should have used double in postgres too.
>>
>> 0.1 decimal is a repeating fraction in binary.
>
> ooops, missed my third point. if precise decimal numbers are
> important, use a decimal data type, which is NUMERIC in SQL (I'm not
> familiar enough with Java to know if it has a BCD or other decimal type).
>
Java BigDecimal is the best fit for Numeric in PostgreSQL

It requires a bit more programming effort, but is the most precise type
for money work.

For instance to sum 2.00 plus 2.00 program:

public static void main(String [] args) {
BigDecimal num1 = new BigDecimal("2.00");
BigDecimal num2 = new BigDecimal("2.00");
BigDecimal sum = num1.add(num2);

if(sum.compareTo(new BigDecimal("4.00"))==0) {
System.out.println("Matches: 2.00+2.00 == 4.00");
} else {
System.out.println("Uau, how did you get here?");
}
}

You must use "compareTo" method to compare two values, and can't use
BigDecimal "equals" to compare values, because 2.0 is different than
2.00 (scale differ)...

Regards,

Edson

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Adrian Klaver 2013-02-25 02:44:14 Re: Floating point error
Previous Message John R Pierce 2013-02-25 02:26:33 Re: Floating point error

Browse pgsql-hackers by date

  From Date Subject
Next Message Adrian Klaver 2013-02-25 02:44:14 Re: Floating point error
Previous Message John R Pierce 2013-02-25 02:26:33 Re: Floating point error