Re: logical replication: patch to ensure timely cleanup of aborted transactions in ReorderBuffer

Hi Vaijayanti,

On Fri, Dec 6, 2024 at 2:50 PM Vaijayanti Bharadwaj
<vaijayanti(dot)bharadwaj(at)enterprisedb(dot)com> wrote:
>
> Hello,
>
> This patch is a proposed fix for an issue that can occur in logical replication where:
> 1. streaming of transactions is disabled.
> 2. First, say there is a transaction that's PREPAREd. This prepared transaction is not resolved for some time. It holds back the oldestRunningXid.
> 3. Large transactions with xid higher than that of the prepared transaction spill to disk as they are decoded in the ReorderBuffer.
> 4. There is an unclean shutdown due to which the large transactions get aborted. But there is no explicit abort record.
> 5. On startup, the ReorderBuffer continues to decode and hold the aborted transactions and they continue to accumulate as spill files.
> 6. Once the prepared transaction is resolved, these transactions will be cleaned up, but if there is a delay, the aborted transactions could hold up considerable space.
> (A customer had run into this issue at EDB)
>
> 0001: is a TAP test that reproduces this and tests for this. It fails without this and passes with it.
> 0002: is the patch that fixes this issue.

A similar problem was discussed at [1]. That thread seems to have
halted without a commit. Can you please check if the solution there
works for you? If so review the patch and help move the thread ahead.

[1] https://www.postgresql.org/message-id/flat/CAD21AoDht9Pz_DFv_R2LqBTBbO4eGrpa9Vojmt5z5sEx3XwD7A%40mail.gmail.com

--
Best Wishes,
Ashutosh Bapat