postgres_fdw: batch inserts vs. before row triggers

From: Etsuro Fujita <etsuro(dot)fujita(at)gmail(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Cc: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>
Subject: postgres_fdw: batch inserts vs. before row triggers
Date: 2022-04-17 09:20:48
Message-ID: CAPmGK16_uPqsmgK0-LpLSUk54_BoK13bPrhxhfjSoSTVz414hA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,
(I added Tomas in CC:.)

One thing I noticed while reviewing the patch for fast copying into
foreign tables/partitions using batch insert [1] is that in
postgres_fdw we allow batch-inserting into foreign tables/partitions
with before row triggers, but such triggers might query the target
table/partition and act differently if the tuples that have already
been processed and prepared for batch-insertion are not there. Here
is an example using HEAD:

create extension postgres_fdw;
create server loopback foreign data wrapper postgres_fdw options
(dbname 'postgres');
create user mapping for current_user server loopback;
create table t (a int);
create foreign table ft (a int) server loopback options (table_name 't');
create function ft_rowcount_tf() returns trigger as $$ begin raise
notice '%: rows = %', tg_name, (select count(*) from ft); return new;
end; $$ language plpgsql;
create trigger ft_rowcount before insert on ft for each row execute
function ft_rowcount_tf();

insert into ft select i from generate_series(1, 10) i;
NOTICE: ft_rowcount: rows = 0
NOTICE: ft_rowcount: rows = 1
NOTICE: ft_rowcount: rows = 2
NOTICE: ft_rowcount: rows = 3
NOTICE: ft_rowcount: rows = 4
NOTICE: ft_rowcount: rows = 5
NOTICE: ft_rowcount: rows = 6
NOTICE: ft_rowcount: rows = 7
NOTICE: ft_rowcount: rows = 8
NOTICE: ft_rowcount: rows = 9
INSERT 0 10

This looks good, but when batch insert is enabled, the trigger
produces incorrect results:

alter foreign table ft options (add batch_size '10');
delete from ft;

insert into ft select i from generate_series(1, 10) i;
NOTICE: ft_rowcount: rows = 0
NOTICE: ft_rowcount: rows = 0
NOTICE: ft_rowcount: rows = 0
NOTICE: ft_rowcount: rows = 0
NOTICE: ft_rowcount: rows = 0
NOTICE: ft_rowcount: rows = 0
NOTICE: ft_rowcount: rows = 0
NOTICE: ft_rowcount: rows = 0
NOTICE: ft_rowcount: rows = 0
NOTICE: ft_rowcount: rows = 0
INSERT 0 10

So I think we should disable batch insert in such cases, just as we
disable multi insert when there are any before row triggers on the
target (local) tables/partitions in copyfrom.c. Attached is a patch
for that.

Best regards,
Etsuro Fujita

[1] https://www.postgresql.org/message-id/bc489202-9855-7550-d64c-ad2d83c24867%40postgrespro.ru

Attachment Content-Type Size
postgres_fdw-disable-batching.patch application/octet-stream 4.0 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Zhihong Yu 2022-04-17 10:34:55 Re: Defer selection of asynchronous subplans until the executor initialization stage
Previous Message Etsuro Fujita 2022-04-17 08:49:55 Re: Defer selection of asynchronous subplans until the executor initialization stage