Re: Invalid memory allocation error with pg_recvlogical or with libPQ logical connection

Thanks, I'll take a look.
Does that mean that bytea data cannot be stored in the same row of of data
size >=500 MB? If that is the case we should document this clearly for the
end user.

Regards,
Jitesh

On Mon, Mar 13, 2023 at 5:13 PM Arne Roland <A(dot)Roland(at)index(dot)de> wrote:

> Hello Jitesh Kumar,
>
> without having a deeper look, it looks like there is something exceeding
> our 1 GB limit (after encoding).
>
> I usually try to stay below 500 MB, so I can safely base16 encode my data.
> That makes administration way easier in general.
> When I need to store larger bytea, I usually split it up into multiple
> parts and store them into separate rows.
>
> I assume without further investigation, that this is related to the 1 GB
> field size, documented here:
> https://www.postgresql.org/docs/current/limits.html
>
> I understand this documentation is probably not very helpful to you. Maybe
> we can improve on the docs somewhere.
>
> Regards
> Arne
> ------------------------------
> *From:* jitesh tiwari <jitesh120(at)gmail(dot)com>
> *Sent:* Monday, March 13, 2023 08:01
> *To:* pgsql-bugs(at)postgresql(dot)org <pgsql-bugs(at)postgresql(dot)org>
> *Subject:* Invalid memory allocation error with pg_recvlogical or with
> libPQ logical connection
>
> Hi all,
>
> I have a use case in postgresql where I have inserted 1 row in a table
> with 3 columns as described below.
> create table xyz (
> id int not null primary key,
> col1 bytea,
> col2 text
> );
>
> The table has 1 row with approx 700 MB of data. The bytea column data in
> the row has around 500MB of data while the text column has data of around
> 150MB approximately.
> When I try to fetch the rows with the logical decoding using the
> test_decoding plugin with libPQ connection, I get the below error -
>
> ERROR: invalid memory alloc request size 1073741825
>
> A similar error I see with pg_recvlogical tool.
> pg_recvlogical: error: unexpected termination of replication stream:
> ERROR:
> invalid memory alloc request size 1073741825
> CONTEXT: slot "elob5121_a818598eea3b7a71", output plugin "test_decoding",
> in
> the change callback, associated LSN 1/BC5681E0
> pg_recvlogical: disconnected; waiting 5 seconds to try again.
>
> I have enabled the Server to debug log and I see a similar error -
> 2023-03-01 07:50:05.769 UTC [33486] DETAIL: There are no running
> transactions.
> 2023-03-01 07:50:05.769 UTC [33486] STATEMENT: START_REPLICATION SLOT
> "elob5121_a818598eea3b7a71" LOGICAL 1/9A163D88 ("include-timestamp",
> "include-xids")
> 2023-03-01 07:50:08.122 UTC [33486] ERROR: invalid memory alloc request
> size
> 1073741825
>
> Please suggest if it is a known issue or limitation in postgresql backend
> code. If so please point to the documentation link for the same. If there
> is any workaround as well please update me.
>
> Regards,
> Jitesh Kumar
>