Re: POC: postgres_fdw insert batching

On Sat, Jan 16, 2021 at 12:00 AM tsunakawa(dot)takay(at)fujitsu(dot)com
<tsunakawa(dot)takay(at)fujitsu(dot)com> wrote:
> From: Amit Langote <amitlangote09(at)gmail(dot)com>
> > Okay, so maybe not moving the whole logic into the FDW's
> > BeginForeignModify(), but at least if we move this...
> >
> > @@ -441,6 +449,72 @@ ExecInsert(ModifyTableState *mtstate,
> > + /*
> > + * Determine if the FDW supports batch insert and determine the
> > batch
> > + * size (a FDW may support batching, but it may be disabled for the
> > + * server/table). Do this only once, at the beginning - we don't want
> > + * the batch size to change during execution.
> > + */
> > + if (resultRelInfo->ri_FdwRoutine->GetForeignModifyBatchSize &&
> > + resultRelInfo->ri_FdwRoutine->ExecForeignBatchInsert &&
> > + resultRelInfo->ri_BatchSize == 0)
> > + resultRelInfo->ri_BatchSize =
> > +
> > resultRelInfo->ri_FdwRoutine->GetForeignModifyBatchSize(resultRelInfo);
> >
> > ...into ExecInitModifyTable(), ExecInsert() only needs the following block:
>
> Does ExecInitModifyTable() know all leaf partitions where the tuples produced by VALUES or SELECT go? ExecInsert() doesn't find the target leaf partition for the first time through the call to ExecPrepareTupleRouting()? Leaf partitions can have different batch_size settings.

Good thing you reminded me that this is about inserts, and in that
case no, ExecInitModifyTable() doesn't know all leaf partitions, it
only sees the root table whose batch_size doesn't really matter. So
it's really ExecInitRoutingInfo() that I would recommend to set
ri_BatchSize; right after this block:

/*
* If the partition is a foreign table, let the FDW init itself for
* routing tuples to the partition.
*/
if (partRelInfo->ri_FdwRoutine != NULL &&
partRelInfo->ri_FdwRoutine->BeginForeignInsert != NULL)
partRelInfo->ri_FdwRoutine->BeginForeignInsert(mtstate, partRelInfo);

Note that ExecInitRoutingInfo() is called only once for a partition
when it is initialized after being inserted into for the first time.

For a non-partitioned targets, I'd still say set ri_BatchSize in
ExecInitModifyTable().

--
Amit Langote
EDB: http://www.enterprisedb.com