Re: Limitation relates to memory allocation

On Mon, 14 Oct 2024 at 19:03, Ekaterina Kiryanova
<e(dot)kiryanova(at)postgrespro(dot)ru> wrote:
> Our research showed that the limit is imposed by the palloc() function,
> regardless of whether it is a tuple or not, and if the data is
> serialized or dumped, the effective limit can be even lower, typically
> around 512MB per row. So for allocations exceeding 1GB, the
> palloc_extended() function can be used. Please correct me if I'm wrong.

I think it would be nice to document the row length limitation and
also add a caveat to the "field size" row to mention that outputting
bytea columns larger than 512MB can be problematic and storing values
that size or above is best avoided.

I don't think wording like: "The practical limit is less than 1 GB" is
going to be good enough as it's just not specific enough. The other
places that talk about practical limits on that page are mostly there
because it's likely impossible that anyone could actually reach the
actual limit. For example, 2^32 databases is likely a limit that
nobody would be able to get close. It's pretty easy to hit the bytea
limit, however:

postgres=# create table b (a bytea);
CREATE TABLE
Time: 2.634 ms
postgres=# insert into b values(repeat('a',600*1024*1024)::bytea);
INSERT 0 1
Time: 9725.320 ms (00:09.725)
postgres=# \o out.txt
postgres=# select * from b;
ERROR: invalid memory alloc request size 1258291203
Time: 209.082 ms

that took me about 10 seconds, so I disagree storing larger bytea
values is impractical.

David