pg_upgrade with -j shows no perf improvement

From: Lonni J Friedman <netllama(at)gmail(dot)com>
To: "pgsql-admin(at)postgresql(dot)org" <pgsql-admin(at)postgresql(dot)org>
Subject: pg_upgrade with -j shows no perf improvement
Date: 2013-06-14 21:29:24
Message-ID: CAP=oouF=fAspj-mzFRj0_3bJh9-iNSgfFfc2ONPgE+Y-a5PF7w@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

Greetings,
I'm in the early stages of preparing to upgrade a production 9.2
cluster to 9.3, by testing the beta of 9.3. All of my testing is
happening on RHEL6-x86_64 on a dedicated server with 128GB RAM and 2x
Intel Xeon E5-2670 CPUs, with all of $PGDATA residing on an 8 disk
RAID10 array.

Currently, a full pg_basebackup of my data is approaching 800GB in
size (uncompressed), so this isn't a tiny, trivial database.

I was curious about how much of a performance gain I'd get from
upgrading with the new -j option to pg_upgrade, so first I performed
the upgrade without it to get a baseline. The command I ran for the
upgrade is as follows:
time pg_upgrade -v -d /var/lib/pgsql/9.2/data -D
/var/lib/pgsql/9.3/data -b /usr/pgsql-9.2/bin -B /usr/pgsql-9.3/bin

time reported the following afterward the upgrade had completed successfully:
real 24m59.255s
user 0m17.069s
sys 15m25.153s

I then repeated the upgrade (after blowing away $PGDATA, and running
initdb again for 9.3), and re-ran pg_upgrade with the same command as
above, only with '-j4' appended to the end. Surprisingly, the
completion time was less than 30 seconds faster. I repeated a third
time with '-j8', and that was about the same completion time as with
'-j4'. I guess I could repeat with 'j2', but I'd be surprised if it
was dramatically faster when -j4 was only marginally so. It seems
like the parallelism of the -j option doesn't seem to be helping much
at all, in my case.

Is this expected, or is it possible that there's a bug somewhere? Let
me know if I can provide any logs from the upgrade.

thanks

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Michael Paquier 2013-06-15 01:46:16 Re: [GENERAL] Can't increase shared_buffers for PostgreSQL on openSUSE 12.3
Previous Message Andreas 2013-06-14 20:47:16 Re: Can't increase shared_buffers for PostgreSQL on openSUSE 12.3