Zero-padding and zero-masking fixes for to_char(float)

From: Bruce Momjian <bruce(at)momjian(dot)us>
To: PostgreSQL-development <pgsql-hackers(at)postgreSQL(dot)org>
Subject: Zero-padding and zero-masking fixes for to_char(float)
Date: 2015-03-18 21:52:44
Message-ID: 20150318215244.GA19931@momjian.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

In September, while researching the to_char() buffer overflow bugs fixed
in 9.4.1 (commit 0150ab567bcf5e5913e2b62a1678f84cc272441f), I found an
inconsistency in how to_char() does zero-padding for float4/8 values.
Now that 9.4.1 is released and I am home for a while, I am ready to
address this.

For example, to_char(int4) properly pads with trailing zeros, e.g.

SELECT to_char(int4 '1999999999', '9999999999999999D' || repeat('9', 1000));
------
1999999999.000000000000000000000000000000...

Numeric does the same thing:

SELECT to_char(numeric '99999999999', '9999999999999999D' || repeat('9', 1000));
------
99999999999.00000000000000000000000000000...

However, float4/8 do not supply the requested zero padding:

SELECT to_char(float4 '99999999999', '9999999999999999D' || repeat('9', 1000));
------
99999997952

and

SELECT to_char(float8 '99999999999', '9999999999999999D' || repeat('9', 1000));
------
99999999999.0000

float4/8 are padding to the internal precision, while int4/numeric are
padding based on the requested precision. This is inconsistent.

The first attached patch fixes this, and also zeros the "junk" digits
which exceed the precision of the underlying type:

SELECT to_char(float4 '99999999999', '9999999999999999D' || repeat('9', 1000));
------
99999900000.00000000000000000000000000000...

SELECT to_char(float8 '99999999999', '9999999999999999D' || repeat('9', 1000));
------
99999999999.0000000000000000000000000000....

This "junk" digit zeroing matches the Oracle behavior:

SELECT to_char(1.123456789123456789123456789d, '9.9999999999999999999999999999999999999') as x from dual;
------
1.1234567891234568000000000000000000000

Our output with the patch would be:

SELECT to_char(float8 '1.123456789123456789123456789', '9.9999999999999999999999999999999999999');
------
1.1234567891234500000000000000000000000

which is pretty close.

The second patch adds regression tests for these.

I would like to apply this for 9.5 while I remember what I was doing,
but I guess now that I have written this email, I will be able to keep
it for 9.6 if people prefer.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

Attachment Content-Type Size
to_char_no_junk_digits.diff text/x-diff 5.6 KB
to_char_regression_test.diff text/x-diff 19.0 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2015-03-18 22:16:14 Re: Using 128-bit integers for sum, avg and statistics aggregates
Previous Message Andres Freund 2015-03-18 21:36:52 Re: parallel mode and parallel contexts