From: | Sam Mason <sam(at)samason(dot)me(dot)uk> |
---|---|
To: | pgsql-general(at)postgresql(dot)org |
Subject: | Re: rounding problems |
Date: | 2008-05-14 14:38:17 |
Message-ID: | 20080514143817.GF1657@frubble.xen.chris-lamb.co.uk |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
On Tue, May 13, 2008 at 02:36:18PM -0400, Justin wrote:
> Double holds 15 places which is the highest value of precision it can
> maintain before rounding occurs.
>
> Is is limit less no, but what is?
>
> Practically speaking taking a vale 0.000,000,000,000,001 aka
> 1 trillionth of anything,
But remember that if you add this value onto a large number and then
take off the large number the result will be zero.
(0.000,000,000,01 + 1,000,000) - 1,000,000 ==> 0
0.000,000,000,01 + (1,000,000 - 1,000,000) ==> 0.000,000,000,01
In general, operations on floating point numbers will increase their
errors.
> i view the problem solved for 98% of problems.
Floating point math is good for most problems, hence why most languages
expose the abstraction.
Sam
From | Date | Subject | |
---|---|---|---|
Next Message | Karsten Hilbert | 2008-05-14 14:45:14 | Re: ranked subqueries vs distinct question |
Previous Message | Decibel! | 2008-05-14 14:35:10 | Re: ranked subqueries vs distinct question |