Re: FDW and parallel execution

On 04.04.2017 13:29, Kyotaro HORIGUCHI wrote:
> Hi,
>
> At Sun, 02 Apr 2017 16:30:24 +0300, Konstantin Knizhnik <k(dot)knizhnik(at)postgrespro(dot)ru> wrote in <58E0FCF0(dot)2070603(at)postgrespro(dot)ru>
>> Hi hackers and personally Robet (you are the best expert in both
>> areas).
>> I want to ask one more question concerning parallel execution and FDW.
>> Below are two plans for the same query (TPC-H Q5): one for normal
>> tables, another for FDW to vertical representation of the same data.
>> FDW supports analyze function and is expected to produce the similar
>> statistic as for original tables.
> <big explain>
>> The plans look very similar, but first one is parallel and second -
>> not.
>> My FDW provides implementation for IsForeignScanParallelSafe which
>> returns true.
>> I wonder what can prevent optimizer from using parallel plan in this
>> case?
> Parallel execution requires partial paths. It's the work for
> GetForeignPaths of your FDW.

Thank you very much for explanation.
But unfortunately I still do not completely understand what kind of
queries allow parallel execution with FDW.

Section "FDW Routines for Parallel Execution" of FDW specification says:
> A ForeignScan node can, optionally, support parallel execution. A
> parallel ForeignScan will be executed in multiple processes and should
> return each row only once across all cooperating processes. To do
> this, processes can coordinate through fixed size chunks of dynamic
> shared memory. This shared memory is not guaranteed to be mapped at
> the same address in every process, so pointers may not be used. The
> following callbacks are all optional in general, but required if
> parallel execution is to be supported.

I provided IsForeignScanParallelSafe, EstimateDSMForeignScan,
InitializeDSMForeignSca and InitializeWorkerForeignScan in my FDW.
IsForeignScanParallelSafe returns true.
Also in GetForeignPaths function I created path with
baserel->consider_parallel == true.
Is it enough or I should do something else?

But unfortunately I failed to find any query: sequential scan, grand
aggregation, aggregation with group by, joins... when parallel execution
plan is used for this FDW.
Also there are no examples of using this functions in Postgres
distributive and I failed to find any such examples in Internet.

Can somebody please clarify my situation with parallel execution and FDW
and may be point at some examples?
Thank in advance.

--
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company