| From: | Nathan Bossart <nathandbossart(at)gmail(dot)com> |
|---|---|
| To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
| Cc: | Andres Freund <andres(at)anarazel(dot)de>, Corey Huinker <corey(dot)huinker(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Jeff Davis <pgsql(at)j-davis(dot)com>, Michael Paquier <michael(at)paquier(dot)xyz>, jian he <jian(dot)universality(at)gmail(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com>, Magnus Hagander <magnus(at)hagander(dot)net>, Stephen Frost <sfrost(at)snowman(dot)net>, Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, alvherre(at)alvh(dot)no-ip(dot)org |
| Subject: | Re: Statistics Import and Export |
| Date: | 2025-03-06 19:33:13 |
| Message-ID: | Z8n4eUTS_iEq3mxn@nathan |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Thu, Mar 06, 2025 at 01:47:34PM -0500, Tom Lane wrote:
> 1. pg_upgrade has made a policy judgement to apply parallelism across
> databases not within a database, ie it will launch concurrent dump/
> restore tasks in different DBs but not authorize any one of them to
> eat multiple CPUs. That needs to be re-thought probably, as I think
> that decision dates to before we had useful parallelism in pg_dump and
> pg_restore. I wonder if we could just rip out pg_upgrade's support
> for DB-level parallelism, which is not terribly pretty anyway, and
> simply pass the -j switch straight to pg_dump and pg_restore.
That would certainly help for clusters with one big database with many LOs
or something, but I worry it would hurt the many database case quite a bit.
Maybe we could add a --jobs-per-db option that indicates how to parallelize
dump/restore. If you set --jobs=8 --jobs-per-db=8, the databases would be
dumped serially, but pg_dump would get -j8. If you set --jobs=8 and
--jobs-per-db=2, we'd process 4 databases at a time, each with -j2.
--
nathan
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Tom Lane | 2025-03-06 19:33:30 | Re: ZStandard (with dictionaries) compression support for TOAST compression |
| Previous Message | Noah Misch | 2025-03-06 19:30:13 | Re: Back-patch of: avoid multiple hard links to same WAL file after a crash |