Re: float output precision questions

"Pedro M. Ferreira" <pfrazao(at)ualg(dot)pt> writes:
> Have two parameters, say DOUBLE_OUTPUT and EXTRA_DIGITS. DOUBLE_OUTPUT
> would select from decimal output or normalized output. EXTRA_DIGITS
> would add the required extra digits, from 0 (default) to 3, when output
> is decimal.

I'm not happy with adding the hex-output option, since it's not
very portable and doesn't seem necessary to solve the problem anyway.

But I think an EXTRA_DIGITS setting might be interesting. In
particular, suppose we allowed EXTRA_DIGITS to be negative? Setting
it to -1 or -2 would go a long way towards eliminating our problems
with platform variations in the geometry regression test.

Perhaps something like

extra_float_digits int range -2 to 2, default 0

extra_float_digits adjusts the number of digits displayed for float4 and
float8 output; the base value of 0 means we output FLT_DIG or DBL_DIG
digits respectively.

Per discussion, there's no reason to allow a value greater than 2, but
I'm not as sure what the lower limit should be --- maybe there's some
use in setting it less than -2?

regards, tom lane