Checkpointer crashes with "PANIC: could not fsync file "pg_tblspc/.."

While testing the below case with the hot standby setup (with the
latest code), I have noticed that the checkpointer process crashed
with the $subject error. As per my observation, we have registered the
SYNC_REQUEST when inserting some tuple into the table, and later on
ALTER SET TABLESPACE we have registered the SYNC_UNLINK_REQUEST, which
looks fine so far, then I have noticed that only when the standby is
connected the underlying table file w.r.t the old tablespace is
already deleted. Now, in AbsorbFsyncRequests we don't do anything for
the SYNC_REQUEST even though we have SYNC_UNLINK_REQUEST for the same
file, but since the underlying file is already deleted the
checkpointer cashed while processing the SYNC_REQUEST.

I have spent some time on this but could not figure out how the
relfilenodenode file w.r.t. to the old tablespace is getting deleted
and if I disconnect the standby then it is not getting deleted, not
sure how walsender is playing a role in deleting the file even before
checkpointer process the unlink request.

postgres[8905]=# create tablespace tab location
'/home/dilipkumar/work/PG/install/bin/test';
CREATE TABLESPACE
postgres[8905]=# create tablespace tab1 location
'/home/dilipkumar/work/PG/install/bin/test1';
CREATE TABLESPACE
postgres[8905]=# create database test tablespace tab;
CREATE DATABASE
postgres[8905]=# \c test
You are now connected to database "test" as user "dilipkumar".
test[8912]=# create table t( a int PRIMARY KEY,b text);
CREATE TABLE
test[8912]=# insert into t values (generate_series(1,10), 'aaa');
INSERT 0 10
test[8912]=# alter table t set tablespace tab1 ;
ALTER TABLE
test[8912]=# CHECKPOINT ;
WARNING: 57P02: terminating connection because of crash of another
server process

log shows:
PANIC: could not fsync file
"pg_tblspc/16384/PG_15_202112131/16386/16387": No such file or
directory

backtrace:
#0 0x00007f2f865ff387 in raise () from /lib64/libc.so.6
#1 0x00007f2f86600a78 in abort () from /lib64/libc.so.6
#2 0x0000000000b13da3 in errfinish (filename=0xcf283f "sync.c", ..
#3 0x0000000000978dc7 in ProcessSyncRequests () at sync.c:439
#4 0x00000000005949d2 in CheckPointGuts (checkPointRedo=67653624,
flags=108) at xlog.c:9590
#5 0x00000000005942fe in CreateCheckPoint (flags=108) at xlog.c:9318
#6 0x00000000008a80b7 in CheckpointerMain () at checkpointer.c:444

Note: This smaller test case is derived from one of the bigger
scenarios raised by Neha Sharma [1]

[1]https://www.postgresql.org/message-id/CANiYTQs0E8TcB11eU0C4eNN0tUd%3DSQqsqEtL1AVZP1%3DEnD-49A%40mail.gmail.com

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com