Re: rounding problems

From: Justin <justin(at)emproshunts(dot)com>
To: Sam Mason <sam(at)samason(dot)me(dot)uk>, pgsql-general(at)postgresql(dot)org
Subject: Re: rounding problems
Date: 2008-05-14 15:47:52
Message-ID: 482B09A8.4090806@emproshunts.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

I have forgotten how much i hate C++

Its not doing what you say it would but it did do other odd ball
things. I miss my foxpro :-(.

Plus its not holding 15 precision points

#include <stdio.h>
#include <cmath>

int main()
{
double a = 0.1;
//double b = 1000;
double c = 100000000;
double d ; //= a * b;
for( int i = 1 ; i < 10 ; i++)
{
d = pow(a,i)+ c ;
printf("%.10f\n", d);
d = d-c ;
printf("%.10f\n", d);
}

return 0;
}

Sam Mason wrote:
> On Tue, May 13, 2008 at 02:36:18PM -0400, Justin wrote:
>
>> Double holds 15 places which is the highest value of precision it can
>> maintain before rounding occurs.
>>
>> Is is limit less no, but what is?
>>
>> Practically speaking taking a vale 0.000,000,000,000,001 aka
>> 1 trillionth of anything,
>>
>
> But remember that if you add this value onto a large number and then
> take off the large number the result will be zero.
>
> (0.000,000,000,01 + 1,000,000) - 1,000,000 ==> 0
> 0.000,000,000,01 + (1,000,000 - 1,000,000) ==> 0.000,000,000,01
>
> In general, operations on floating point numbers will increase their
> errors.
>
>
>> i view the problem solved for 98% of problems.
>>
>
> Floating point math is good for most problems, hence why most languages
> expose the abstraction.
>
>
> Sam
>
>

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Ивайло Гелов 2008-05-14 15:51:00 Re: Strange behaviour - performance decreases after each TRUNCATE
Previous Message Karsten Hilbert 2008-05-14 15:46:02 Re: ranked subqueries vs distinct question