Re: Floating point comparison inconsistencies of the geometric types

On Wed, Jun 1, 2016 at 8:08 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Fri, May 27, 2016 at 6:43 AM, Emre Hasegeli <emre(at)hasegeli(dot)com> wrote:
>> There are those macros defined for the built-in geometric types:
>>
>>> #define EPSILON 1.0E-06
>>
>>> #define FPzero(A) (fabs(A) <= EPSILON)
>>> #define FPeq(A,B) (fabs((A) - (B)) <= EPSILON)
>>> #define FPne(A,B) (fabs((A) - (B)) > EPSILON)
>>> #define FPlt(A,B) ((B) - (A) > EPSILON)
>>> #define FPle(A,B) ((A) - (B) <= EPSILON)
>>> #define FPgt(A,B) ((A) - (B) > EPSILON)
>>> #define FPge(A,B) ((B) - (A) <= EPSILON)
>>
>> with this warning:
>>
>>> * XXX These routines were not written by a numerical analyst.
>
> I agree that those macros looks like a pile of suck.

+1

> It's unclear to
> me what purpose they're trying to accomplish, but regardless of what
> it is, it's hard for me to believe that they are accomplishing it.
> Whether 1.0E-06 is a correct fuzz factor presumably depends greatly on
> the scale of the number; e.g. if the values are all like "3" it's
> probably fine but if they are like "37142816124856" it's probably not
> enough fuzz and if they are all like ".00000004" it's probably way too
> much fuzz.

Also, it's a pretty lame heuristic. It doesn't really eliminate
*any* problems; it just aims to make them less frequent by shifting
the edges around. In doing so, it creates whole new classes of
problems:

test=# \set A '''(1.9999996,1.9999996),(1,1)''::box'
test=# \set B '''(2,2),(1,1)''::box'
test=# \set C '''(2.0000004,2.0000004),(1,1)''::box'
test=# select :A = :B, :B = :C, :A = :C;
?column? | ?column? | ?column?
----------+----------+----------
t | t | f
(1 row)

Is the benefit we get from the macros worth destroying the
transitive property of the comparison operators on these types?

> Figuring out what to do about it is harder. Your proposal seems to be
> to remove them except where we need the fuzzy behavior, which doesn't
> sound unreasonable, but I don't personally understand why we need it
> in some places and not others.

+1

My first inclination is to remove those macros in version 10, but
it would be good to hear from some people using these types on what
the impact of that would be.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company