Re: pg_upgrade failing for 200+ million Large Objects

"Kumar, Sachin" <ssetiya(at)amazon(dot)com> writes:
>> On 11/12/2023, 01:43, "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us <mailto:tgl(at)sss(dot)pgh(dot)pa(dot)us>> wrote:
>> ... Maybe that
>> could be improved in future, but it seems like it'd add a
>> lot more complexity, and it wouldn't make life any better for
>> pg_upgrade (which doesn't use parallel pg_restore, and seems
>> unlikely to want to in future).

> I was not able to find email thread which details why we are not using
> parallel pg_restore for pg_upgrade.

Well, it's pretty obvious isn't it? The parallelism is being applied
at the per-database level instead.

> IMHO most of the customer will have single large
> database, and not using parallel restore will cause slow pg_upgrade.

You've offered no justification for that opinion ...

> I am attaching a patch which enables parallel pg_restore for DATA and POST-DATA part
> of dump. It will push down --jobs value to pg_restore and will restore
> database sequentially.

I don't think I trust this patch one bit. It makes way too many
assumptions about how the --section options work, or even that they
will work at all in a binary-upgrade situation. I've spent enough
time with that code to know that --section is pretty close to being
a fiction. One point in particular is that this would change the
order of ACL restore relative to other steps, which almost certainly
will cause problems for somebody.

regards, tom lane