Re: pg_restore parallel from machine with more CPUs than the server... bad?

From: Vijaykumar Jain <vijaykumarjain(dot)github(at)gmail(dot)com>
To: Wells Oliver <wells(dot)oliver(at)gmail(dot)com>
Cc: pgsql-admin <pgsql-admin(at)postgresql(dot)org>
Subject: Re: pg_restore parallel from machine with more CPUs than the server... bad?
Date: 2021-06-13 16:42:01
Message-ID: CAM+6J95xXmCQYMKg5U3mkZxC4WW_C3g_SdBHHO+0AgSK+67oQA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

>I have an AWS RDS instance with 4 CPU. I have a EC2 instance I fired up
with 16 CPU thinking I could pg_restore --jobs 16 and it would be great,
and my restore ground to a halt with a bunch of locks on refreshing
materializing views about 600GB into ~700GB.

I am not too sure if it is bad, just that more workers would make use of
more resources mem etc :)
Before a restore, I bump maintenance mem etc based resources to speed up
building indexes etc.
and then restore it to what it was previously set.
it will still be mostly 4 cpu at work :) doing context switching among 16
workers.

if your db has multiple small objects, parallel restore should be faster
than sequential.
If you have many large tables, I am not sure too many workers would be of
great help.

as noted in one old thread,
pg14 introduced parallel support for materialised views. till then even
with parallelism on, it would only use one worker.

>I was wondering if there was any risk associated with restoring on a
computer with more CPU than the destination server (and indeed using more
CPU than the server)?
I do not think so. but if this degrades the performance compared with the
same setting, then you should come back here to let everyone know the same.

In response to

Browse pgsql-admin by date

  From Date Subject
Next Message Tom Lane 2021-06-13 17:11:43 Re: pg_restore: warning: invalid creation date in header
Previous Message Wells Oliver 2021-06-13 16:24:50 pg_restore parallel from machine with more CPUs than the server... bad?