Limitation relates to memory allocation

From: Ekaterina Kiryanova <e(dot)kiryanova(at)postgrespro(dot)ru>
To: pgsql-docs(at)lists(dot)postgresql(dot)org
Subject: Limitation relates to memory allocation
Date: 2024-10-14 06:03:36
Message-ID: 440403b2-5c91-45a8-88c9-db031f05ef4d@postgrespro.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-docs

Hello!

We encountered an issue related to internal memory allocation limit:
ERROR: invalid memory alloc request size

In the documentation on limits:
https://www.postgresql.org/docs/17/limits.html
the description suggests that only a single field is limited to 1GB,
which could imply that the total tuple size can be larger. So as we
planned to store three columns of 1GB each in a table and attempted to
insert this data, we got the error.

Our research showed that the limit is imposed by the palloc() function,
regardless of whether it is a tuple or not, and if the data is
serialized or dumped, the effective limit can be even lower, typically
around 512MB per row. So for allocations exceeding 1GB, the
palloc_extended() function can be used. Please correct me if I'm wrong.

I prepared a small patch for master, if it's worth clarifying, could you
please review the attachment?

--
Ekaterina Kiryanova
Technical Writer
Postgres Professional
the Russian PostgreSQL Company

Attachment Content-Type Size
palloc-limitation.patch text/x-patch 1.0 KB

Responses

Browse pgsql-docs by date

  From Date Subject
Next Message PG Doc comments form 2024-10-14 22:57:22 missing command?
Previous Message Bruce Momjian 2024-10-12 16:37:23 Re: Streaming Replication vs Logical