Re: [DESIGN] ParallelAppend

On Fri, Aug 21, 2015 at 7:40 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>
> On Tue, Aug 18, 2015 at 11:27 PM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
wrote:
> >> Here is one other issue I found. Existing code assumes a TOC segment
has
> >> only one contents per node type, so it uses pre-defined key (like
> >> PARALLEL_KEY_SCAN) per node type, however, it is problematic if we put
> >> multiple PlannedStmt or PartialSeqScan node on a TOC segment.
> >
> > We have few keys in parallel-seq-scan patch
> > (PARALLEL_KEY_TUPLE_QUEUE and PARALLEL_KEY_INST_INFO) for
> > which multiple structures are shared between master and worker backends.
> >
> > Check if something similar can work for your use case.
>
> I think you are possibly missing the point.

It could be possible, but let me summarize what I thought would be required
for above use case. For Parallel Append, we need to push multiple
planned statements in contrast to one planned statement as is done for
current patch and then one or more parallel workers needs to work on each
planned statement. So if we know in advance how many planned statements
are we passing down (which we should), then using ParallelWorkerNumber
(ParallelWorkerNumber % num_planned_statements or some other similar
way), workers can find the the planned statement on which they need to work
and similarly information for PartialSeqScan (which currently is parallel
heap
scan descriptor information).

> I think KaiGai's correct,
> and I pointed out the same problem to you before. The parallel key
> for the Partial Seq Scan needs to be allocated on the fly and carried
> in the node, or we'll never be able to push multiple things below the
> funnel.

Okay, immediately I don't see what is the best way to achieve this but
let us discuss this separately on Parallel Seq Scan thread and let me
know if you have something specific in your mind. I will also give this
a more thought.

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com