From: | "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com> |
---|---|
To: | PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
Subject: | Fdw batch insert error out when set batch_size > 65535 |
Date: | 2021-05-21 02:48:05 |
Message-ID: | OS0PR01MB571603973C0AC2874AD6BF2594299@OS0PR01MB5716.jpnprd01.prod.outlook.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hi,
When reading the code, I noticed some possible issue about fdw batch insert.
When I set batch_size > 65535 and insert more than 65535 rows into foreign table,
It will throw an error:
For example:
------------------
CREATE FOREIGN TABLE vzdalena_tabulka2(a int, b varchar)
SERVER omega_db
OPTIONS (table_name 'tabulka', batch_size '65536');
INSERT INTO vzdalena_tabulka2 SELECT i, 'AHOJ' || i FROM generate_series(1,65536) g(i);
ERROR: number of parameters must be between 0 and 65535
CONTEXT: remote SQL command: INSERT INTO public.tabulka(a, b) VALUES ($1, $2), ($3, $4)...
------------------
Actually, I think if the (number of columns) * (number of rows) > 65535, then we can
get this error. But, I think almost no one will set such a large value, so I think adjust the
batch_size automatically when doing INSERT seems an acceptable solution.
In the postgresGetForeignModifyBatchSize(), if we found the (num of param) * batch_size
Is greater than the limit(65535), then we set it to 65535 / (num of param).
Thoughts ?
Best regards,
houzj
Attachment | Content-Type | Size |
---|---|---|
0001-limit-the-fdw-batch-size.patch | application/octet-stream | 2.9 KB |
0002-doc-note.patch | application/octet-stream | 1.1 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2021-05-21 03:04:51 | Re: multi-install PostgresNode fails with older postgres versions |
Previous Message | Kyotaro Horiguchi | 2021-05-21 02:21:05 | Re: Race condition in recovery? |