Re: BUG #17334: Assert failed inside computeDistance() on gist index scanning

Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> writes:
> FWIW I can reproduce this on master too. The failure happens because of
> NaN value in the index:

Man, what a pain those are.

> I'm not sure if the issue is in allowing the NaN to be added to the
> index, or not handling it correctly during the index scan.

Surely omitting the entry from the index would lead to incorrect
answers. Without any index, we get

regression=# CREATE TABLE point_tbl(f1 point);
CREATE TABLE
regression=# INSERT INTO point_tbl SELECT ('0,0') FROM generate_series(1, 2);
INSERT 0 2
regression=# INSERT INTO point_tbl VALUES ('0,NaN');
INSERT 0 1
regression=# SELECT f1, f1 <-> point '(0,0)' AS dist FROM point_tbl ORDER BY dist;
f1 | dist
---------+------
(0,0) | 0
(0,0) | 0
(0,NaN) | NaN
(3 rows)

You can argue about where the NaN distance should sort, but
not about whether the row should appear at all.

> It's interesting btree_gist does not have issues (for NaN in float8
> columns). It seems not to store NaN in the index, It seems to replace
> them with tiny values, at least according to pageinspect.

Yipes, that's even worse, if true.

regards, tom lane