pgsql: Adjust batch size in postgres_fdw to not use too many parameters

From: Tomas Vondra <tomas(dot)vondra(at)postgresql(dot)org>
To: pgsql-committers(at)lists(dot)postgresql(dot)org
Subject: pgsql: Adjust batch size in postgres_fdw to not use too many parameters
Date: 2021-06-08 18:28:45
Message-ID: E1lqgSz-0004ny-7I@gemulon.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Adjust batch size in postgres_fdw to not use too many parameters

The FE/BE protocol identifies parameters with an Int16 index, which
limits the maximum number of parameters per query to 65535. With
batching added to postges_fdw this limit is much easier to hit, as
the whole batch is essentially a single query, making this error much
easier to hit.

The failures are a bit unpredictable, because it also depends on the
number of columns in the query. So instead of just failing, this patch
tweaks the batch_size to not exceed the maximum number of parameters.

Reported-by: Hou Zhijie <houzj(dot)fnst(at)cn(dot)fujitsu(dot)com>
Reviewed-by: Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>
Discussion: https://postgr.es/m/OS0PR01MB571603973C0AC2874AD6BF2594299%40OS0PR01MB5716.jpnprd01.prod.outlook.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/cb92703384e2bb3fa0a690e5dbb95ad333c2b44c

Modified Files
--------------
contrib/postgres_fdw/expected/postgres_fdw.out | 11 +++++++++++
contrib/postgres_fdw/postgres_fdw.c | 11 +++++++++--
contrib/postgres_fdw/sql/postgres_fdw.sql | 7 +++++++
doc/src/sgml/postgres-fdw.sgml | 11 +++++++++++
src/interfaces/libpq/fe-exec.c | 21 ++++++++++++---------
src/interfaces/libpq/libpq-fe.h | 2 ++
6 files changed, 52 insertions(+), 11 deletions(-)

Browse pgsql-committers by date

  From Date Subject
Next Message Jeremy Schneider 2021-06-08 18:35:57 Re: logical decoding bug: segfault in ReorderBufferToastReplace()
Previous Message Tomas Vondra 2021-06-08 17:38:53 pgsql: Fix pg_visibility regression failure with CLOBBER_CACHE_ALWAYS