pgsql: Fix float-to-integer coercions to handle edge cases correctly.

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: pgsql-committers(at)lists(dot)postgresql(dot)org
Subject: pgsql: Fix float-to-integer coercions to handle edge cases correctly.
Date: 2018-11-24 01:57:17
Message-ID: E1gQNC9-0007RW-HY@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Fix float-to-integer coercions to handle edge cases correctly.

ftoi4 and its sibling coercion functions did their overflow checks in
a way that looked superficially plausible, but actually depended on an
assumption that the MIN and MAX comparison constants can be represented
exactly in the float4 or float8 domain. That fails in ftoi4, ftoi8,
and dtoi8, resulting in a possibility that values near the MAX limit will
be wrongly converted (to negative values) when they need to be rejected.

Also, because we compared before rounding off the fractional part,
the other three functions threw errors for values that really ought
to get rounded to the min or max integer value.

Fix by doing rint() first (requiring an assumption that it handles
NaN and Inf correctly; but dtoi8 and ftoi8 were assuming that already),
and by comparing to values that should coerce to float exactly, namely
INTxx_MIN and -INTxx_MIN. Also remove some random cosmetic discrepancies
between these six functions.

Per bug #15519 from Victor Petrovykh. This should get back-patched,
but first let's see what the buildfarm thinks of it --- I'm not too
sure about portability of some of the regression test cases.

Patch by me; thanks to Andrew Gierth for analysis and discussion.

Discussion: https://postgr.es/m/15519-4fc785b483201ff1@postgresql.org

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/cbdb8b4c0155b2d9679ecd79b886c29c2730b85d

Modified Files
--------------
src/backend/utils/adt/float.c | 79 +++++++++++++++++++---
src/backend/utils/adt/int8.c | 52 +++++++++-----
src/test/regress/expected/float4.out | 49 ++++++++++++++
src/test/regress/expected/float8-small-is-zero.out | 49 ++++++++++++++
src/test/regress/expected/float8.out | 49 ++++++++++++++
src/test/regress/sql/float4.sql | 14 ++++
src/test/regress/sql/float8.sql | 14 ++++
7 files changed, 277 insertions(+), 29 deletions(-)

Browse pgsql-committers by date

  From Date Subject
Next Message Tom Lane 2018-11-24 04:49:33 pgsql: Adjust new test case for more portability.
Previous Message Michael Paquier 2018-11-24 01:35:03 Re: pgsql: Add WL_EXIT_ON_PM_DEATH pseudo-event.