Re: rounding problems

On Wed, May 14, 2008 at 11:47:52AM -0400, Justin wrote:
> I have forgotten how much i hate C++

What we're talking about doesn't have much to do with C++, it's floating
point maths in general.

> Its not doing what you say it would but it did do other odd ball
> things. I miss my foxpro :-(.

What does foxpro use for storing numbers? or is it just that you never
pushed it hard enough for the abstractions to show through.

> Plus its not holding 15 precision points

after changing the output to be:

printf("%.10f %.10f\n", d, d-c);

I get:

100000000.0999999940 0.0999999940
100000000.0100000054 0.0100000054
100000000.0010000020 0.0010000020
100000000.0001000017 0.0001000017
100000000.0000099987 0.0000099987
100000000.0000009984 0.0000009984
100000000.0000001043 0.0000001043
100000000.0000000149 0.0000000149
100000000.0000000000 0.0000000000

Which looks reasonable. Remember that floating point numbers store
their state in base two, not base ten. All of those numbers look good
to 15 decimal digits.

Sam