Re: pgsql: Introduce framework for parallelizing various pg_upgrade tasks.

Hi!

On Tue, Sep 17, 2024 at 12:11 AM Nathan Bossart <nathan(at)postgresql(dot)org> wrote:
> Introduce framework for parallelizing various pg_upgrade tasks.
>
> A number of pg_upgrade steps require connecting to every database
> in the cluster and running the same query in each one. When there
> are many databases, these steps are particularly time-consuming,
> especially since they are performed sequentially, i.e., we connect
> to a database, run the query, and process the results before moving
> on to the next database.
>
> This commit introduces a new framework that makes it easy to
> parallelize most of these once-in-each-database tasks by processing
> multiple databases concurrently. This framework manages a set of
> slots that follow a simple state machine, and it uses libpq's
> asynchronous APIs to establish the connections and run the queries.
> The --jobs option is used to determine the number of slots to use.
> To use this new task framework, callers simply need to provide the
> query and a callback function to process its results, and the
> framework takes care of the rest. A more complete description is
> provided at the top of the new task.c file.
>
> None of the eligible once-in-each-database tasks are converted to
> use this new framework in this commit. That will be done via
> several follow-up commits.
>
> Reviewed-by: Jeff Davis, Robert Haas, Daniel Gustafsson, Ilya Gladyshev, Corey Huinker
> Discussion: https://postgr.es/m/20240516211638.GA1688936%40nathanxps13

Should we add UpgradeTaskProcessCB to the typedefs.list? I don't see
this would directly influence indentation right now, but probably we
should do for uniformity?

------
Regards,
Alexander Korotkov
Supabase