Re: How to estimate the shared memory size required for parallel scan?

Hi Thomas,

Thank you for explaining DSM and ToC.

> It's up to you to design a struct to hold whatever data,
spinlocks, LWLocks, atomics etc you might need to orchestrate your
parallel scan.

If FDW(ex. cstore_fdw) does not need to share some information among
workers more than PostgreSQL core in parallel scan, does it not need
to allocate DSM?

thanks.
2018年8月17日(金) 9:28 Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>:
>
> On Thu, Aug 16, 2018 at 11:36 PM, Masayuki Takahashi
> <masayuki038(at)gmail(dot)com> wrote:
> > I am trying to change cstore_fdw to scan in parallel. FDW interface provide
> > 'EstimateDSMForeignScan' for required shared memory size to scan in parallel.
> >
> > How to estimate the shared memory size required for parallel scan?
>
> It's a slightly strange use of the word "estimate". It means "tell me
> how much shared memory you need". Later, your
> InitializeDSMForeignScan() callback will receive a pointer to exactly
> that much shared memory to initialise. Then
> InitializeWorkerForeignScan will also receive a pointer to that
> memory, inside every worker process. Note that it may be mapped at a
> different address in each process, so be careful not to use raw
> pointers. It's up to you to design a struct to hold whatever data,
> spinlocks, LWLocks, atomics etc you might need to orchestrate your
> parallel scan.
>
> It works much the same way for built-in executor nodes that are
> parallel-aware by the way. For example, ExecHashJoinEstimate()
> reserves sizeof(ParallelHashJoinState), and then in
> ExecHashJoinInitializeDSM() it allocates and initialises it, and
> ExecHashJoinIntializeWorker() tells the workers about it. The
> built-in executor nodes have to do a little bit more work than FDWs,
> using the plan node ID to allocate and look things up in a "TOC"
> (table of contents), but nodeForeignScan.c does that work for you in
> your case: it just asks you how much you want, and then gives it to
> you.
>
> --
> Thomas Munro
> http://www.enterprisedb.com

--
高橋 真之