Failed to delete old ReorderBuffer spilled files

From: atorikoshi <torikoshi_atsushi_z2(at)lab(dot)ntt(dot)co(dot)jp>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Failed to delete old ReorderBuffer spilled files
Date: 2017-11-20 10:35:11
Message-ID: 54e4e488-186b-a056-6628-50628e4e4ebc@lab.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

I put many queries into one transaction and made ReorderBuffer spill
data to disk, and sent SIGKILL to postgres before the end of the
transaction.

After starting up postgres again, I observed the files spilled to
data wasn't deleted.

I think these files should be deleted because its transaction was no
more valid, so no one can use these files.

Below is a reproduction instructions.

------------------------------------------------
1. Create table and publication at publiser.

@pub =# CREATE TABLE t1(
id INT PRIMARY KEY,
name TEXT);

@pub =# CREATE PUBLICATION pub FOR TABLE t1;

2. Create table and subscription at subscriber.

@sub =# CREATE TABLE t1(
id INT PRIMARY KEY,
name TEXT
);

@sub =# CREATE SUBSCRIPTION sub
CONNECTION 'host=[hostname] port=[port] dbname=[dbname]'
PUBLICATION pub;

3. Put many queries into one transaction.

@pub =# BEGIN;
INSERT INTO t1
SELECT
i,
'aaaaaaaaaa'
FROM
generate_series(1, 1000000) as i;

4. Then we can see spilled files.

@pub $ ls -1 ${PGDATA}/pg_replslot/sub/
state
xid-561-lsn-0-1000000.snap
xid-561-lsn-0-2000000.snap
xid-561-lsn-0-3000000.snap
xid-561-lsn-0-4000000.snap
xid-561-lsn-0-5000000.snap
xid-561-lsn-0-6000000.snap
xid-561-lsn-0-7000000.snap
xid-561-lsn-0-8000000.snap
xid-561-lsn-0-9000000.snap

5. Kill publisher's postgres process before COMMIT.

@pub $ kill -s SIGKILL [pid of postgres]

6. Start publisher's postgres process.

@pub $ pg_ctl start -D ${PGDATA}

7. After a while, we can see the files remaining.
(Immediately after starting publiser, we can not see these files.)

@pub $ pg_ctl start -D ${PGDATA}

When I configured with '--enable-cassert', below assertion error
was appeared.

TRAP: FailedAssertion("!(txn->final_lsn != 0)", File:
"reorderbuffer.c", Line: 2576)
------------------------------------------------

Attached patch sets final_lsn to the last ReorderBufferChange if
final_lsn == 0.

--
Atsushi Torikoshi
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

Attachment Content-Type Size
set_final_lsn.patch text/plain 930 bytes

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Dmitry Dolgov 2017-11-20 10:47:04 Re: [HACKERS] [PATCH] Generic type subscripting
Previous Message Ildus Kurbangaliev 2017-11-20 09:44:28 Re: [HACKERS] Custom compression methods