Re: How best to do parallel query given tens of thousands of iteration of a loop of recursive queries?

From: "David G(dot) Johnston" <david(dot)g(dot)johnston(at)gmail(dot)com>
To: Shaozhong SHI <shishaozhong(at)gmail(dot)com>
Cc: pgsql-sql <pgsql-sql(at)lists(dot)postgresql(dot)org>
Subject: Re: How best to do parallel query given tens of thousands of iteration of a loop of recursive queries?
Date: 2022-04-10 16:13:14
Message-ID: CAKFQuwYsSUU1USBTzBjyNFUTzgGbLqoxZyT7o2hhuYFcyfR2CA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-sql

On Sun, Apr 10, 2022 at 6:50 AM Shaozhong SHI <shishaozhong(at)gmail(dot)com>
wrote:

> There is a plpgsql script that have a loop to carry out the same recursive
> queries.
>
> The estimation of iteration is in the order of tens of thousands.
>
> What is the best way of making using parallel query strategy.
>

An example would be helpful.

However, as a general guideline, since parallelism is done at the per-row
scope, removing looping logic from the script and turning the main script
logic into one or more functions that operate on a single row, while
obeying the rules such functions need to abide by in order to be marked
parallel safe, will open up the possibility for the server to process
different rows using different workers and then appending their results
together for the next node to consume.

David J.

In response to

Browse pgsql-sql by date

  From Date Subject
Next Message Voillequin, Jean-Marc 2022-04-12 13:12:25 enable/disable instead of trigger
Previous Message Steve Midgley 2022-04-10 16:03:04 Re: How best to do parallel query given tens of thousands of iteration of a loop of recursive queries?