COPY big row returns ERROR: invalid memory alloc request size

From: Lars Aksel Opsahl <Lars(dot)Opsahl(at)nibio(dot)no>
To: "pgsql-admin(at)lists(dot)postgresql(dot)org" <pgsql-admin(at)lists(dot)postgresql(dot)org>
Subject: COPY big row returns ERROR: invalid memory alloc request size
Date: 2021-04-13 04:58:37
Message-ID: AM7P189MB1028FB16E33E3A87C33C96649D4F9@AM7P189MB1028.EURP189.PROD.OUTLOOK.COM
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

Hi

I have problem with Postgres replication failing and it seems to be related to very big rows.

Here is an example

SELECT rep_id, pg_column_size(version_union_tables_r03.harvested_per_object.*) from version_union_tables_r03.harvested_per_object where rep_id = 778;
rep_id | pg_column_size
--------+----------------
778 | 1064407412

If I create TEMP table it works OK with that row.

CREATE temp table test as (select * from version_union_tables_r03.harvested_per_object where rep_id = 778);
SELECT 1

If I run the COPY cmd it fails

COPY (select * FROM version_union_tables_r03.harvested_per_object where rep_id = 778) TO STDOUT;
ERROR: invalid memory alloc request size 2127087159

Postgres Replication are using the COPY cmd.

Any ideas on how can get around this problem for postgres replication without changing the table def ?

Here is the table def.

Table "version_union_tables_r03.harvested_per_object"
Column | Type | Collation | Nullable | Default
----------------+-----------------------------+-----------+----------+-------------------------------------------------------------------------------
contractor_id | character varying | | |
machine_key | character varying | | |
machine_name | character varying | | |
object_key | integer | | |
object_name | character varying | | |
object_user_id | character varying | | |
start_date | timestamp without time zone | | |
end_date | timestamp without time zone | | |
mean_dbh | double precision | | |
mean_m3_sub | double precision | | |
mean_m3_sob | double precision | | |
mean_length_dm | double precision | | |
n_logs | integer | | |
n_trees | integer | | |
m3_sub | double precision | | |
m3_sob | double precision | | |
ba_m2 | double precision | | |
doc_mod_date | timestamp without time zone | | |
area_m2 | integer | | |
lat | double precision | | |
lon | double precision | | |
geom | geometry | | |
rep_id | integer | | not null | nextval('version_union_tables_r03.harvested_per_object_rep_id_seq'::regclass)
Indexes:
"harvested_per_object_pkey" PRIMARY KEY, btree (rep_id)
"version_union_tables_r03_harvested_harvested_per_object_machine" btree (machine_key, object_key)
Publications:
"version_union_tables_r03_pub"

Thanks

Lars

Browse pgsql-admin by date

  From Date Subject
Next Message Goti 2021-04-13 06:00:10 ERROR: column c.relhasoids does not exist in Postgres 13
Previous Message Henry Francisco Garcia Cortez 2021-04-12 14:51:18 Re: autovacuum_vacuum_cost_limit