Re: [POC] Fast COPY FROM command for the table with foreign partitions

On Tue, Dec 1, 2020 at 2:40 PM tsunakawa(dot)takay(at)fujitsu(dot)com
<tsunakawa(dot)takay(at)fujitsu(dot)com> wrote:
> From: Amit Langote <amitlangote09(at)gmail(dot)com>
> > Andrey's original patch had the flag to, as I understand it, make the
> > partitioning case work correctly. When inserting into a
> > non-partitioned table, there's only one relation to care about. In
> > that case, CopyFrom() can use either the new COPY interface or the
> > INSERT interface for the entire operation when talking to a foreign
> > target relation's FDW driver. With partitions, that has to be
> > considered separately for each partition. What complicates the matter
> > further is that while the original target relation (the root
> > partitioned table in the partitioning case) is fully initialized in
> > CopyFrom(), partitions are lazily initialized by ExecFindPartition().
>
> Yeah, I felt it a bit confusing to see the calls to Begin/EndForeignInsert() in both CopyFrom() and ExecInitRoutingInfo().
>
> > Note that the initialization of a given target relation can also
> > optionally involve calling the FDW to perform any pre-COPY
> > initializations. So if a given partition is a foreign table, whether
> > the copy operation was initialized using the COPY interface or the
> > INSERT interface is determined away from CopyFrom(). Andrey created
> > ri_usesMultiInsert to remember which was used so that CopyFrom() can
> > use the correct interface during the subsequent interactions with the
> > partition's driver.
> >
> > Now, it does not seem outright impossible to do this without the flag,
> > but maybe Andrey thinks it is good for readability? If it is
> > confusing from a modularity standpoint, maybe we should rethink that.
> > That said, I still think that there should be a way for CopyFrom() to
> > tell ExecFindPartition() which FDW interface to initialize a given
> > foreign table partition's copy operation with -- COPY if the copy
> > allows multi-insert, INSERT if not. Maybe the multi_insert parameter
> > I mentioned earlier would serve that purpose.
>
> I agree with your idea of adding multi_insert argument to ExecFindPartition() to request a multi-insert-capable partition. At first, I thought ExecFindPartition() is used for all operations, insert/delete/update/select, so I found it odd to add multi_insert argument. But ExecFindPartion() is used only for insert, so multi_insert argument seems okay.

Good. Andrey, any thoughts on this?

--
Amit Langote
EDB: http://www.enterprisedb.com