logical replication: patch to ensure timely cleanup of aborted transactions in ReorderBuffer

From: Vaijayanti Bharadwaj <vaijayanti(dot)bharadwaj(at)enterprisedb(dot)com>
To: pgsql-hackers(at)lists(dot)postgresql(dot)org
Cc: Vaijayanti Bharadwaj <vaijayantib(at)gmail(dot)com>
Subject: logical replication: patch to ensure timely cleanup of aborted transactions in ReorderBuffer
Date: 2024-12-06 09:20:01
Message-ID: CAFNcJWUaHGzRk6qJe-tm3qQ9XLEMfj9-Prksk15uACMf8qQ=BA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello,

This patch is a proposed fix for an issue that can occur in logical
replication where:
1. streaming of transactions is disabled.
2. First, say there is a transaction that's PREPAREd. This prepared
transaction is not resolved for some time. It holds back the
oldestRunningXid.
3. Large transactions with xid higher than that of the prepared transaction
spill to disk as they are decoded in the ReorderBuffer.
4. There is an unclean shutdown due to which the large transactions get
aborted. But there is no explicit abort record.
5. On startup, the ReorderBuffer continues to decode and hold the aborted
transactions and they continue to accumulate as spill files.
6. Once the prepared transaction is resolved, these transactions will be
cleaned up, but if there is a delay, the aborted transactions could hold up
considerable space.
(A customer had run into this issue at EDB)

0001: is a TAP test that reproduces this and tests for this. It fails
without this and passes with it.
0002: is the patch that fixes this issue.

Thanks,
Vaijayanti

Attachment Content-Type Size
0001-TAP-test-that-passes-with-the-ReorderBuffer-cleanup-.patch application/octet-stream 4.7 KB
0002-cleanup-incomplete-transactions-on-startup-in-Reorde.patch application/octet-stream 5.1 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Devanga.Susmitha@fujitsu.com 2024-12-06 09:29:29 Re: Popcount optimization using SVE for ARM
Previous Message Amul Sul 2024-12-06 09:13:51 Re: NOT ENFORCED constraint feature