Re: interval_ops shall stop using btequalimage (deduplication)

On Tue, Oct 10, 2023 at 6:33 PM Noah Misch <noah(at)leadboat(dot)com> wrote:
> interval_ops, however, recognizes equal-but-distinguishable values:

> Fails with:
>
> 2498151 2023-10-10 05:06:46.177 GMT DEBUG: building index "ti" on table "t" serially
> 2498151 2023-10-10 05:06:46.178 GMT DEBUG: index "ti" can safely use deduplication
> TRAP: failed Assert("!itup_key->allequalimage || keepnatts == _bt_keep_natts_fast(rel, lastleft, firstright)"), File: "nbtutils.c", Line: 2443, PID: 2498151

Nice catch.

Out of curiosity, how did you figure this out? Did it just occur to
you that interval_ops had a behavior that made deduplication unsafe?
Or did the problem come to your attention after running amcheck on a
customer database? Or was it something else?

> I've also caught btree posting lists where one TID refers to a '1d' heap
> tuple, while another TID refers to a '24h' heap tuple. amcheck complains.
> Index-only scans can return the '1d' bits where the actual tuple had the '24h'
> bits. Are there other consequences to highlight in the release notes?

Nothing else comes to mind right now. I should think about posting
list splits some more tomorrow, though -- those are tricky.

> The back-branch patch is larger, to fix things without initdb. Hence, I'm
> attaching patches for HEAD and for v16 (trivial to merge back from there).

> I glanced at the other opfamilies permitting deduplication, and they look okay:

Due to the way that nbtsplitloc.c deals with duplicates, I'd expect
the same assertion failure with any index where a single leaf page is
filled with opclass-wise duplicates with more than one distinct
representation/output -- the details beyond that shouldn't matter. I
was happy with how easy it was to make this assertion fail (with a
known broken numeric_ops opclass) while testing/developing
deduplication. I'm a little surprised that it took this long to notice
the interval_ops issue.

Do we really need to change the catalog contents when backpatching?

--
Peter Geoghegan