fetching bytea (blob) data of 850 MB from psql client failed

From: jitesh tiwari <jitesh120(at)gmail(dot)com>
To: pgsql-bugs(at)postgresql(dot)org
Subject: fetching bytea (blob) data of 850 MB from psql client failed
Date: 2022-08-22 07:14:17
Message-ID: CAOEdJKF4PFrHtH0OEn+Qmeqy-Z7UnNWUMAtrT+jVhTo3ozUjxw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

*Hi all, *

*I have a use case in postgresql where I have inserted 500 rows in a table
with 2 columns as described below. *
*create table xyz (*
*id citext not null primary key, *
*col1 bytea*
*);*

*The table has 500 rows and each row has around 850 MB of data. The bytea
column data in the row has around 830 to 840 MB of data while citext column
data has 10 to to 15 bytes approximately.*
*When I try to fetch all rows with a select query or try to fetch a
single row with a select statement using the where clause for this
table, the Database throws the below error -*

*VERROR; invalid memory alloc request size 1764311653(File mcxt.c; Line
959;Routine palloc; )*

*The error above pointing the PostgreSQL database backend code.*
*May I know why I am getting this error? If I use array fetch using the
ODBC driver with fetchsize=1 (Assuming the application will fetch 1 record
at a time from the PostgreSQL database server) then also I get that error.
Is there any Server configuration which can control this **memory**
allocation error and allow me to fetch one record at time from the
PostgreSQL database table?*

*Please suggest if it is a known issue or limitation in postgresql backend
code. If so please point to the documentation link for the same. If there
is any workaround as well please update me.*

Regards,
Jitesh

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Zhang Mingli 2022-08-22 07:45:11 Re: BUG #17591: elog(ERROR) cause SharedSnapshotLock deadlock
Previous Message PG Bug reporting form 2022-08-22 06:23:59 BUG #17591: elog(ERROR) cause SharedSnapshotLock deadlock