Re: Zero-padding and zero-masking fixes for to_char(float)

On Tue, Mar 24, 2015 at 09:47:56AM -0400, Noah Misch wrote:
> On Sun, Mar 22, 2015 at 10:53:12PM -0400, Bruce Momjian wrote:
> > On Sun, Mar 22, 2015 at 04:41:19PM -0400, Noah Misch wrote:
> > > On Wed, Mar 18, 2015 at 05:52:44PM -0400, Bruce Momjian wrote:
> > > > This "junk" digit zeroing matches the Oracle behavior:
> > > >
> > > > SELECT to_char(1.123456789123456789123456789d, '9.9999999999999999999999999999999999999') as x from dual;
> > > > ------
> > > > 1.1234567891234568000000000000000000000
> > > >
> > > > Our output with the patch would be:
> > > >
> > > > SELECT to_char(float8 '1.123456789123456789123456789', '9.9999999999999999999999999999999999999');
> > > > ------
> > > > 1.1234567891234500000000000000000000000
>
> > > These outputs show Oracle treating 17 digits as significant while PostgreSQL
> > > treats 15 digits as significant. Should we match Oracle in this respect while
> > > we're breaking compatibility anyway? I tend to think yes.
> >
> > Uh, I am hesistant to adjust our precision to match Oracle as I don't
> > know what they are using internally.
>
> http://sqlfiddle.com/#!4/8b4cf/5 strongly implies 17 significant digits for
> float8 and 9 digits for float4.

I was able to get proper rounding with the attached patch.

test=> SELECT to_char(float8 '1.123456789123456789123456789', '9.9999999999999999999999999999999999999');
to_char
------------------------------------------
1.1234567891234600000000000000000000000
(1 row)

Handling rounding for exponent-format values turned out to be simple.
What has me stuck now is how to do rounding in the non-decimal part of
the number, e.g.

test=> SELECT to_char(float4 '15555555555555.912345678912345678900000000000000000000000',
repeat('9', 50) || '.' || repeat('9', 50));
to_char
--------------------------------------------------------------------------------------------------------
15555555753984.00000000000000000000000000000000000000000000000000
(1 row)

This should return something like 15555600000000.000... (per Oracle
output at the URL above, float4 has 6 significant digits on my compiler)
but I can't seem to figure how to get printf() to round non-fractional
parts. I am afraid the only solution is to use printf's %e format and
place the decimal point myself.

The fact I still don't have a complete solution suggests this is 9.6
material but I still want to work on it so it is ready.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +