Memory leak in pg_logical_slot_{get,peek}_changes

From: vignesh C <vignesh21(at)gmail(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Memory leak in pg_logical_slot_{get,peek}_changes
Date: 2024-12-11 04:27:38
Message-ID: CALDaNm1hewNAsZ_e6FF52a=9drmkRJxtEPrzCB6-9mkJyeBBqA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

I'm starting a new thread for one of the issues reported by Sawada-san at [1].

This is a memory leak on CacheMemoryContext when using pgoutput via SQL APIs:
/* Map must live as long as the session does. */
oldctx = MemoryContextSwitchTo(CacheMemoryContext);

entry->attrmap = build_attrmap_by_name_if_req(indesc, outdesc, false);

MemoryContextSwitchTo(oldctx);
RelationClose(ancestor);

entry->attrmap is pfree'd only when validating the RelationSyncEntry
so remains even after logical decoding API calls.

This issue can be reproduced with the following test:
-- Create table
create table t1( c1 int, c2 int, c3 int) PARTITION BY LIST (c1);
create table t1_1( c2 int, c1 int, c3 int);
ALTER TABLE t1 ATTACH PARTITION t1_1 FOR VALUES IN (0, 1, 2, 3);

-- Create publication
create publication pub1 FOR TABLE t1, t1_1 WITH
(publish_via_partition_root = true);

-- Create replication slot
SELECT 'init' FROM pg_create_logical_replication_slot('test', 'pgoutput');

-- Run the below steps between Test start and Test end many times to
see the memory usage getting increased
-- Test start
insert into t1 values( 1);
SELECT count(*) FROM pg_logical_slot_get_binary_changes('test', NULL,
NULL, 'proto_version', '4', 'publication_names', 'pub1');

-- Memory increases after each invalidation and insert
SELECT * FROM pg_backend_memory_contexts where name = 'CacheMemoryContext';
-- Test end

The attached patch resolves a memory leak by ensuring that the
attribute map is properly freed during plugin shutdown. This process
is triggered by the SQL API when the decoding context is being
released. Additionally, I am conducting a review to identify and
address any similar memory leaks that may exist elsewhere in the code.

[1] - https://www.postgresql.org/message-id/CAD21AoDkAhQVSukOfH3_reuF-j4EU0-HxMqU3dU%2BbSTxsqT14Q%40mail.gmail.com

Regards,
Vignesh

Attachment Content-Type Size
v1-0001-Fix-memory-leak-in-pgoutput-relation-attribute-ma.patch text/x-patch 1.4 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2024-12-11 04:38:31 Re: attndims, typndims still not enforced, but make the value within a sane threshold
Previous Message Shlok Kyal 2024-12-11 03:43:32 Re: long-standing data loss bug in initial sync of logical replication