| From: | Etsuro Fujita <etsuro(dot)fujita(at)gmail(dot)com> | 
|---|---|
| To: | PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> | 
| Cc: | Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> | 
| Subject: | postgres_fdw: batch inserts vs. before row triggers | 
| Date: | 2022-04-17 09:20:48 | 
| Message-ID: | CAPmGK16_uPqsmgK0-LpLSUk54_BoK13bPrhxhfjSoSTVz414hA@mail.gmail.com | 
| Views: | Whole Thread | Raw Message | Download mbox | Resend email | 
| Thread: | |
| Lists: | pgsql-hackers | 
Hi,
(I added Tomas in CC:.)
One thing I noticed while reviewing the patch for fast copying into
foreign tables/partitions using batch insert [1] is that in
postgres_fdw we allow batch-inserting into foreign tables/partitions
with before row triggers, but such triggers might query the target
table/partition and act differently if the tuples that have already
been processed and prepared for batch-insertion are not there.  Here
is an example using HEAD:
create extension postgres_fdw;
create server loopback foreign data wrapper postgres_fdw options
(dbname 'postgres');
create user mapping for current_user server loopback;
create table t (a int);
create foreign table ft (a int) server loopback options (table_name 't');
create function ft_rowcount_tf() returns trigger as $$ begin raise
notice '%: rows = %', tg_name, (select count(*) from ft); return new;
end; $$ language plpgsql;
create trigger ft_rowcount before insert on ft for each row execute
function ft_rowcount_tf();
insert into ft select i from generate_series(1, 10) i;
NOTICE:  ft_rowcount: rows = 0
NOTICE:  ft_rowcount: rows = 1
NOTICE:  ft_rowcount: rows = 2
NOTICE:  ft_rowcount: rows = 3
NOTICE:  ft_rowcount: rows = 4
NOTICE:  ft_rowcount: rows = 5
NOTICE:  ft_rowcount: rows = 6
NOTICE:  ft_rowcount: rows = 7
NOTICE:  ft_rowcount: rows = 8
NOTICE:  ft_rowcount: rows = 9
INSERT 0 10
This looks good, but when batch insert is enabled, the trigger
produces incorrect results:
alter foreign table ft options (add batch_size '10');
delete from ft;
insert into ft select i from generate_series(1, 10) i;
NOTICE:  ft_rowcount: rows = 0
NOTICE:  ft_rowcount: rows = 0
NOTICE:  ft_rowcount: rows = 0
NOTICE:  ft_rowcount: rows = 0
NOTICE:  ft_rowcount: rows = 0
NOTICE:  ft_rowcount: rows = 0
NOTICE:  ft_rowcount: rows = 0
NOTICE:  ft_rowcount: rows = 0
NOTICE:  ft_rowcount: rows = 0
NOTICE:  ft_rowcount: rows = 0
INSERT 0 10
So I think we should disable batch insert in such cases, just as we
disable multi insert when there are any before row triggers on the
target (local) tables/partitions in copyfrom.c.  Attached is a patch
for that.
Best regards,
Etsuro Fujita
[1] https://www.postgresql.org/message-id/bc489202-9855-7550-d64c-ad2d83c24867%40postgrespro.ru
| Attachment | Content-Type | Size | 
|---|---|---|
| postgres_fdw-disable-batching.patch | application/octet-stream | 4.0 KB | 
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Zhihong Yu | 2022-04-17 10:34:55 | Re: Defer selection of asynchronous subplans until the executor initialization stage | 
| Previous Message | Etsuro Fujita | 2022-04-17 08:49:55 | Re: Defer selection of asynchronous subplans until the executor initialization stage |