Re: float output precision questions

On Wed, 30 Oct 2002, Tom Lane wrote:

> Stephan Szabo <sszabo(at)megazone23(dot)bigpanda(dot)com> writes:
> > On Wed, 30 Oct 2002, Pedro M. Ferreira wrote:
> >> I looked at some of these emails and it seemed to me that the problem
> >> was that Tom did'nt want a parameter that would force people to know
> >> about printf number formatting. I think the first solution above (the
> >> SHORT and LONG way) is simple, maintains usual output as default and
> >> enables 'maximum' precision at request.
>
> > That seems reasonable then, Tom'll probably give any other objections he
> > might have if he has any.
>
> My recollection is that other people (perhaps Peter?) were the ones
> objecting before. However I'd be somewhat unhappy with the proposal
> as given:
>
> >>Option 'SHORT' would be default and produce the standard sprintf(ascii,...
> >>Option 'LONG' would produce sprintf(ascii, "%25.18g", num).
>
> since this seems to me to hardwire inappropriate assumptions about the
> number of significant digits in a double. (Yes, I know practically
> everyone uses IEEE floats these days. But it's inappropriate for PG
> to assume that.)

True (which I actually was trying to get at in my messages as well). I'll
admit to having not read the precise proposal. It's really pretty outside
what I work with in any case.

> AFAICT the real issue here is that binary float representations will
> have a fractional decimal digit of precision beyond what DBL_DIG claims.
> I think I could support adding an option that switches between the
> current output format:
> sprintf(ascii, "%.*g", DBL_DIG, num);
> and:
> sprintf(ascii, "%.*g", DBL_DIG+1, num);
> and similarly for float4. Given carefully written float I/O routines,
> reading the latter output should reproduce the originally stored value.
> (And if the I/O routines are not carefully written, you probably lose
> anyway.) I don't see a need for allowing more flexibility than that.

Well, on my system, it doesn't look like doing the above sprintfs will
actually work for all numbers. I did a simple program using an arbitrary
big number and the DBL_DIG+1 output when stuck into another double
actually was a different double value. DBL_DIG+2 worked on my system,
but...