Re: Proposal for resolving casting issues

> > > > Have you seen my example ? If calculated in float4 the result of
> > > > 1.00000000000001*1000.0-1000.0 would be 0.0, no ?
> > >
> > > So? If you are storing one input as float4, then you cannot rationally
> > > say that you know the result to better than 6 digits, because you don't
> > > know the input to better than 6 digits. Claiming that 1000.00000000001
> > > is a more accurate answer for the product than 1000.0 is simply wishful
> > > thinking on your part: nothing to the right of the sixth digit actually
> > > means a darn thing, because you don't know whether the input was really
> > > exactly 1000, or should have been perhaps 1000.001.
> >
> > I still see 1E-10 as a better answer to above calculation than your 0,
> > and my snapshot 9/11 does return that 1E-10.
>
> Well, then you'd be wrong. Numerical analysis says you _can't_ get more
> information out than went in to the _least_ precise part of a calculation.
> What your suggesting is the equivalent of wanting to put up a shelf, so
> you estimate the length of the wall by eyeballing it, then measure the
> wood for the shelf with a micrometer, to be sure it fits
> exactly right.

> We teach this in intro science classes all the time: if you calculate with
> 3.14 as an approximation to pi, you better not report the circumference
> of a circle as 2.45678932 cm, I'll take off points!

What if he must display 9 digits and says the result is approximately 2.45678932
would that be worse than 2.46000000 ?
That is what I am trying to say. Probably the standard is meant as a hint for db
users, that such results will be approximate, not where the first digit sits that
is not exact any more.

For above calculation pg will in the future return 0.00000000000000000000 as an
answer to 1.00000000000001*1000.0-1000.0 when used in my example context, while
it currently returns 0.000000000010 ...
You both are saying, that 0.00000000000000000000 is a better answer.

Andreas