Re: TRAP: FailedAssertion("tabstat->trans == trans", File: "pgstat_relation.c", Line: 508

Op 19-04-2022 om 11:25 schreef Kyotaro Horiguchi:
> Thaks Erik.
>
> At Tue, 19 Apr 2022 07:00:30 +0200, Erik Rijkers <er(at)xs4all(dot)nl> wrote in
>> Op 19-04-2022 om 02:15 schreef Kyotaro Horiguchi:
>>> Could you read tabstat, *tabstat, trans, *trans here?
>>
>> To be honest I'm not sure how to, but I gave it a try:
>>

I rebuilt newest master (a62bff74b135) with

export CUSTOM_COPT="-O0 -g"

The 12th run of statbug.sh crashed and gave a corefile.

GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from
/home/aardvark/pg_stuff/pg_installations/pgsql.HEAD/bin/postgres...done.
[New LWP 25058]

warning: Can't read pathname for load map: Input/output error.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `postgres: logical replication worker for
subscription 16411 '.
Program terminated with signal 6, Aborted.
#0 0x000000357d6324f5 in raise () from /lib64/libc.so.6
(gdb) bt
#0 0x000000357d6324f5 in raise () from /lib64/libc.so.6
#1 0x000000357d633cd5 in abort () from /lib64/libc.so.6
#2 0x0000000000b3bada in ExceptionalCondition (conditionName=0xd389a1
"tabstat->trans == trans", errorType=0xd388b2 "FailedAssertion",
fileName=0xd388a0 "pgstat_relation.c", lineNumber=508) at assert.c:69
#3 0x00000000009bf5dc in AtEOXact_PgStat_Relations
(xact_state=0x31b1b50, isCommit=true) at pgstat_relation.c:508
#4 0x00000000009c4107 in AtEOXact_PgStat (isCommit=true,
parallel=false) at pgstat_xact.c:54
#5 0x0000000000583764 in CommitTransaction () at xact.c:2360
#6 0x0000000000584354 in CommitTransactionCommand () at xact.c:3048
#7 0x000000000090b34e in apply_handle_commit_internal
(commit_data=0x7ffd024b5940) at worker.c:1532
#8 0x000000000090a287 in apply_handle_commit (s=0x7ffd024b59b0) at
worker.c:845
#9 0x000000000090ce3a in apply_dispatch (s=0x7ffd024b59b0) at worker.c:2473
#10 0x000000000090d41c in LogicalRepApplyLoop (last_received=74680880)
at worker.c:2757
#11 0x000000000090e974 in start_apply (origin_startpos=0) at worker.c:3526
#12 0x000000000090f156 in ApplyWorkerMain (main_arg=0) at worker.c:3782
#13 0x00000000008c7623 in StartBackgroundWorker () at bgworker.c:858
#14 0x00000000008d1557 in do_start_bgworker (rw=0x30ff0a0) at
postmaster.c:5802
#15 0x00000000008d1903 in maybe_start_bgworkers () at postmaster.c:6026
#16 0x00000000008d09ba in sigusr1_handler (postgres_signal_arg=10) at
postmaster.c:5191
#17 <signal handler called>
#18 0x000000357d6e1683 in __select_nocancel () from /lib64/libc.so.6
#19 0x00000000008cc6c1 in ServerLoop () at postmaster.c:1757
#20 0x00000000008cc0aa in PostmasterMain (argc=11, argv=0x30d6590) at
postmaster.c:1465
#21 0x00000000007c9256 in main (argc=11, argv=0x30d6590) at main.c:202
(gdb) f 3
#3 0x00000000009bf5dc in AtEOXact_PgStat_Relations
(xact_state=0x31b1b50, isCommit=true) at pgstat_relation.c:508
508 Assert(tabstat->trans == trans);
(gdb) p tabstat
$1 = (PgStat_TableStatus *) 0x319e630
(gdb) p *tabstat
$2 = {t_id = 2139062143, t_shared = 127, trans = 0x7f7f7f7f7f7f7f7f,
t_counts = {t_numscans = 9187201950435737471, t_tuples_returned =
9187201950435737471, t_tuples_fetched = 9187201950435737471,
t_tuples_inserted = 9187201950435737471, t_tuples_updated =
9187201950435737471, t_tuples_deleted = 9187201950435737471,
t_tuples_hot_updated = 9187201950435737471, t_truncdropped = 127,
t_delta_live_tuples = 9187201950435737471, t_delta_dead_tuples =
9187201950435737471, t_changed_tuples = 9187201950435737471,
t_blocks_fetched = 9187201950435737471, t_blocks_hit = 9187201950435737471},
relation = 0x7f7f7f7f7f7f7f7f}
(gdb) p trans
$3 = (PgStat_TableXactStatus *) 0x31b1ba8
(gdb) p *trans
$4 = {tuples_inserted = 1, tuples_updated = 0, tuples_deleted = 0,
truncdropped = false, inserted_pre_truncdrop = 0, updated_pre_truncdrop
= 0, deleted_pre_truncdrop = 0, nest_level = 1, upper = 0x0,
parent = 0x319e630, next = 0x31b1ab8}
(gdb)

Looks like we're one step further, no?

Erik

>> (gdb) p tabstat
>> $1 = <optimized out>
>
> Great! It is that. But unfortunately they are optimized out.. Could
> you cause the crash with -O0 binary? You will see the variable with
> it. You can rebuild with the option as follows.
>
> $ make clean; make install CUSTOM_COPT="-O0 -g"
>
> You can dump only the whole xact_state chain from the current core
> file but the result will give a bit obscure hint for diagnosis.
>
> regards.
>