From: | Emanuel Calvo <postgres(dot)arg(at)gmail(dot)com> |
---|---|
To: | pgsql-hackers(at)postgresql(dot)org |
Subject: | Parallel pg_basebackup |
Date: | 2013-08-23 13:51:28 |
Message-ID: | CAGHEX6a9uhK7RtjTwRoW0Tp27dRwcrOFo19Hvw6VAYZeHNZ5fA@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hi guys,
I was wondering if there is a proposal for parallelize pg_basebackup (sorry
if there is already one, I looked into the inbox and did not find any).
I made a simple test and the performance gain is pretty considerable using
parallel rsync processes vs. single pg_basebackup.
i.e., for 46GB of data I had the following results:
Using https://gist.github.com/rcoup/5358786:
rsync, parallel, including the xlogs
real 18m52.674s
user 33m54.728s
sys 13m44.604s
(-x option)
real 23m26.802s
user 0m24.392s
sys 3m55.116s
Actually, AWS services limits the bandwidth per process but if you have
parallel processes, none of them will have the IOPS decreased which will be
translated in a great improvement. Also the network bandwidth is strong
between the regions.
Thanks guys!
--
--
Emanuel Calvo
From | Date | Subject | |
---|---|---|---|
Next Message | Greg Stark | 2013-08-23 13:58:51 | Re: Does larger i/o size make sense? |
Previous Message | Amit Kapila | 2013-08-23 13:11:04 | Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review]) |