From: | Justin <justin(at)emproshunts(dot)com> |
---|---|
To: | Sam Mason <sam(at)samason(dot)me(dot)uk>, pgsql-general(at)postgresql(dot)org |
Subject: | Re: rounding problems |
Date: | 2008-05-14 15:47:52 |
Message-ID: | 482B09A8.4090806@emproshunts.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
I have forgotten how much i hate C++
Its not doing what you say it would but it did do other odd ball
things. I miss my foxpro :-(.
Plus its not holding 15 precision points
#include <stdio.h>
#include <cmath>
int main()
{
double a = 0.1;
//double b = 1000;
double c = 100000000;
double d ; //= a * b;
for( int i = 1 ; i < 10 ; i++)
{
d = pow(a,i)+ c ;
printf("%.10f\n", d);
d = d-c ;
printf("%.10f\n", d);
}
return 0;
}
Sam Mason wrote:
> On Tue, May 13, 2008 at 02:36:18PM -0400, Justin wrote:
>
>> Double holds 15 places which is the highest value of precision it can
>> maintain before rounding occurs.
>>
>> Is is limit less no, but what is?
>>
>> Practically speaking taking a vale 0.000,000,000,000,001 aka
>> 1 trillionth of anything,
>>
>
> But remember that if you add this value onto a large number and then
> take off the large number the result will be zero.
>
> (0.000,000,000,01 + 1,000,000) - 1,000,000 ==> 0
> 0.000,000,000,01 + (1,000,000 - 1,000,000) ==> 0.000,000,000,01
>
> In general, operations on floating point numbers will increase their
> errors.
>
>
>> i view the problem solved for 98% of problems.
>>
>
> Floating point math is good for most problems, hence why most languages
> expose the abstraction.
>
>
> Sam
>
>
From | Date | Subject | |
---|---|---|---|
Next Message | Ивайло Гелов | 2008-05-14 15:51:00 | Re: Strange behaviour - performance decreases after each TRUNCATE |
Previous Message | Karsten Hilbert | 2008-05-14 15:46:02 | Re: ranked subqueries vs distinct question |