Some memory not freed at the exit of RelationBuildPartitionDesc()

From: amul sul <sulamul(at)gmail(dot)com>
To: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Some memory not freed at the exit of RelationBuildPartitionDesc()
Date: 2019-08-08 07:14:23
Message-ID: CAAJ_b952=5fLxfDS3rd01w9D=wM9OTSowrXuOZdG41dYDk56Dw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

In RelationBuildPartitionDesc(), a memory space that use to gather
partitioning
bound info wasn't free at the end. This might not a problem because this
allocated memory will eventually be recovered when the top-level context is
freed, but the case when a partitioned table having 1000s or more
partitions and
this partitioned relation open & close, and its cached entry invalidated in
loop
then we'll have too may call to RelationBuildPartitionDesc() which will keep
wasting some space with every loop.

For a demonstration purpose, I did the following changes to
heap_drop_with_catalog() and tried to drop a partitioned table having 5000
partitions(attached create script) which hit OOM on a machine in no time:

diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index b7bcdd9d0f..6b7bc0d7ae 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -1842,6 +1842,8 @@ heap_drop_with_catalog(Oid relid)
parentOid = get_partition_parent(relid);
LockRelationOid(parentOid, AccessExclusiveLock);

+ rel = relation_open(parentOid, NoLock);
+ relation_close(rel, NoLock);
/*
* If this is not the default partition, dropping it will change the
* default partition's partition constraint, so we must lock it.

I think we should do all partitioned bound information gathering and
calculation in temporary memory context which can be released at the end of
RelationBuildPartitionDesc(), thoughts/Comments?

I did the same in the attached patch and the aforesaid OOM issue is
disappeared.

Regards,
Amul

Attachment Content-Type Size
temp_MemoryContext_POC.patch application/octet-stream 1.5 KB
create_5k_partitions.sql application/octet-stream 281.1 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Yonatan Misgan 2019-08-08 07:30:36 Locale support
Previous Message Konstantin Knizhnik 2019-08-08 07:03:34 Re: Global temporary tables