BUG #18490: Assert in comparetup_index_btree_tiebreak() fails when pg_class reindexed during a table creation

The following bug has been logged on the website:

Bug reference: 18490
Logged by: Alexander Lakhin
Email address: exclusion(at)gmail(dot)com
PostgreSQL version: 17beta1
Operating system: Ubuntu 22.04
Description:

The following script:
for ((i=1;i<=10000;i++)); do
createdb db
psql -c "CREATE TABLE tbl1 (id serial primary key, t text)" -d db >>log1 &
psql -c "REINDEX INDEX pg_class_tblspc_relfilenode_index" -d db >>log2
wait
grep 'TRAP' server.log && { echo "on iteration $i"; break; }
dropdb db
done

triggers an assert:
TRAP: failed Assert("false"), File: "tuplesortvariants.c", Line: 1582, PID:
3119702
on iteration 2147

Core was generated by `postgres: law db [local] REINDEX
'.
Program terminated with signal SIGABRT, Aborted.

(gdb) bt
#0 __pthread_kill_implementation (no_tid=0, signo=6,
threadid=140427932190528) at ./nptl/pthread_kill.c:44
#1 __pthread_kill_internal (signo=6, threadid=140427932190528) at
./nptl/pthread_kill.c:78
#2 __GI___pthread_kill (threadid=140427932190528, signo=signo(at)entry=6) at
./nptl/pthread_kill.c:89
#3 0x00007fb7ec427476 in __GI_raise (sig=sig(at)entry=6) at
../sysdeps/posix/raise.c:26
#4 0x00007fb7ec40d7f3 in __GI_abort () at ./stdlib/abort.c:79
#5 0x00005590c7bb8dda in ExceptionalCondition (conditionName=0x5590c7e2c9f7
"false", fileName=0x5590c7e2c78e "tuplesortvariants.c", lineNumber=1582) at
assert.c:66
#6 0x00005590c7c14a81 in comparetup_index_btree_tiebreak (a=0x5590c9a23660,
b=0x5590c9a23648, state=0x5590c9a1a620) at tuplesortvariants.c:1582
#7 0x00005590c7c145c4 in comparetup_index_btree (a=0x5590c9a23660,
b=0x5590c9a23648, state=0x5590c9a1a620) at tuplesortvariants.c:1462
#8 0x00005590c7c0b890 in qsort_tuple (data=0x5590c9a23570, n=11,
compare=0x5590c7c1453e <comparetup_index_btree>, arg=0x5590c9a1a620) at
../../../../src/include/lib/sort_template.h:354
#9 0x00005590c7c0bb2e in qsort_tuple (data=0x5590c9a21488, n=165,
compare=0x5590c7c1453e <comparetup_index_btree>, arg=0x5590c9a1a620) at
../../../../src/include/lib/sort_template.h:405
#10 0x00005590c7c1036f in tuplesort_sort_memtuples (state=0x5590c9a1a620) at
tuplesort.c:2759
#11 0x00005590c7c0d7b3 in tuplesort_performsort (state=0x5590c9a1a620) at
tuplesort.c:1406
#12 0x00005590c7495051 in _bt_leafbuild (btspool=0x5590c9972830,
btspool2=0x0) at nbtsort.c:551
#13 0x00005590c7494b68 in btbuild (heap=0x7fb7ed0cefd8,
index=0x7fb7e05d0288, indexInfo=0x5590c99744b8) at nbtsort.c:328
#14 0x00005590c753bf92 in index_build (heapRelation=0x7fb7ed0cefd8,
indexRelation=0x7fb7e05d0288, indexInfo=0x5590c99744b8, isreindex=true,
parallel=true) at index.c:3017
#15 0x00005590c753d31f in reindex_index (stmt=0x5590c9948268, indexId=3455,
skip_constraint_checks=false, persistence=112 'p', params=0x7ffe61b6a054) at
index.c:3754
#16 0x00005590c764e2a8 in ReindexIndex (stmt=0x5590c9948268,
params=0x7ffe61b6a0c8, isTopLevel=true) at indexcmds.c:2820
#17 0x00005590c764e0ad in ExecReindex (pstate=0x5590c9971be0,
stmt=0x5590c9948268, isTopLevel=true) at indexcmds.c:2743
#18 0x00005590c79c3e15 in ProcessUtilitySlow (pstate=0x5590c9971be0,
pstmt=0x5590c9948318, queryString=0x5590c99477c0 "REINDEX INDEX
pg_class_tblspc_relfilenode_index", context=PROCESS_UTILITY_TOPLEVEL,
params=0x0, queryEnv=0x0,
dest=0x5590c99486d8, qc=0x7ffe61b6a8d0) at utility.c:1567
#19 0x00005590c79c2c27 in standard_ProcessUtility (pstmt=0x5590c9948318,
queryString=0x5590c99477c0 "REINDEX INDEX
pg_class_tblspc_relfilenode_index", readOnlyTree=false,
context=PROCESS_UTILITY_TOPLEVEL, params=0x0, queryEnv=0x0,
dest=0x5590c99486d8, qc=0x7ffe61b6a8d0) at utility.c:1067
#20 0x00005590c79c1b28 in ProcessUtility (pstmt=0x5590c9948318,
queryString=0x5590c99477c0 "REINDEX INDEX
pg_class_tblspc_relfilenode_index", readOnlyTree=false,
context=PROCESS_UTILITY_TOPLEVEL, params=0x0, queryEnv=0x0,
dest=0x5590c99486d8, qc=0x7ffe61b6a8d0) at utility.c:523
#21 0x00005590c79c03a3 in PortalRunUtility (portal=0x5590c99c6cb0,
pstmt=0x5590c9948318, isTopLevel=true, setHoldSnapshot=false,
dest=0x5590c99486d8, qc=0x7ffe61b6a8d0) at pquery.c:1158
#22 0x00005590c79c061a in PortalRunMulti (portal=0x5590c99c6cb0,
isTopLevel=true, setHoldSnapshot=false, dest=0x5590c99486d8,
altdest=0x5590c99486d8, qc=0x7ffe61b6a8d0) at pquery.c:1315
#23 0x00005590c79bfa64 in PortalRun (portal=0x5590c99c6cb0,
count=9223372036854775807, isTopLevel=true, run_once=true,
dest=0x5590c99486d8, altdest=0x5590c99486d8, qc=0x7ffe61b6a8d0) at
pquery.c:791
#24 0x00005590c79b8720 in exec_simple_query (query_string=0x5590c99477c0
"REINDEX INDEX pg_class_tblspc_relfilenode_index") at postgres.c:1274
#25 0x00005590c79bd7c5 in PostgresMain (dbname=0x5590c9980cb0 "db",
username=0x5590c9980c98 "law") at postgres.c:4680
#26 0x00005590c79b4713 in BackendMain (startup_data=0x7ffe61b6ab54 "",
startup_data_len=4) at backend_startup.c:105
#27 0x00005590c78c9cc2 in postmaster_child_launch (child_type=B_BACKEND,
startup_data=0x7ffe61b6ab54 "", startup_data_len=4,
client_sock=0x7ffe61b6aba0) at launch_backend.c:265
#28 0x00005590c78d0145 in BackendStartup (client_sock=0x7ffe61b6aba0) at
postmaster.c:3593
#29 0x00005590c78cd205 in ServerLoop () at postmaster.c:1674
#30 0x00005590c78ccacc in PostmasterMain (argc=1, argv=0x5590c9941760) at
postmaster.c:1372
#31 0x00005590c7779a99 in main (argc=1, argv=0x5590c9941760) at main.c:197

(gdb) p *((IndexTuple) a->tuple)
$1 = {t_tid = {ip_blkid = {bi_hi = 0, bi_lo = 0}, ip_posid = 3}, t_info =
16}
(gdb) p *((IndexTuple) b->tuple)
$2 = {t_tid = {ip_blkid = {bi_hi = 0, bi_lo = 0}, ip_posid = 3}, t_info =
16}

With debug logging added, I confirmed that on a higher lever, inside
btbuild(), buildstate.spool->sortstate->memtuples really contains two
tuples pointing to blkno 0, offset 3 after the _bt_spools_heapscan() call.

Reproduced on REL_10_STABLE .. master.