From: | Suraj Kharage <suraj(dot)kharage(at)enterprisedb(dot)com> |
---|---|
To: | Robert Haas <robertmhaas(at)gmail(dot)com> |
Cc: | Rushabh Lathia <rushabh(dot)lathia(at)gmail(dot)com>, Ahsan Hadi <ahsan(dot)hadi(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, David Zhang <david(dot)zhang(at)highgo(dot)ca>, Asif Rehman <asifr(dot)rehman(at)gmail(dot)com>, Kashif Zeeshan <kashif(dot)zeeshan(at)enterprisedb(dot)com>, Rajkumar Raghuwanshi <rajkumar(dot)raghuwanshi(at)enterprisedb(dot)com>, Jeevan Chalke <jeevan(dot)chalke(at)enterprisedb(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: WIP/PoC for parallel backup |
Date: | 2020-05-22 06:03:29 |
Message-ID: | CAF1DzPUyTSWgZRmpGm=0GHkTT8rSVVS-rZcKv+Nv7ZaueS-Hzg@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Thu, May 21, 2020 at 7:12 PM Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>
> So, basically, when we go from 1 process to 4, the additional
> processes spend all of their time waiting rather than doing any useful
> work, and that's why there is no performance benefit. Presumably, the
> reason they spend all their time waiting for ClientRead/ClientWrite is
> because the network between the two machines is saturated, so adding
> more processes that are trying to use it at maximum speed just leads
> to spending more time waiting for it to be available.
>
> Do we have the same results for the local backup case, where the patch
> helped?
>
Here is the result for local backup case (100GB data). Attaching the
captured logs.
The total number of events (pg_stat_activity) captured during local runs:
- 82 events for normal backups
- 31 events for parallel backups (-j 4)
BaseBackupRead wait event numbers: (newly added)
24 - in normal backups
14 - in parallel backup (-j 4)
ClientWrite wait event numbers:
8 - in normal backup
43 - in parallel backups
ClientRead wait event numbers:
0 - ClientRead in normal backup
32 - ClientRead in parallel backups for diff processes.
--
--
Thanks & Regards,
Suraj kharage,
EnterpriseDB Corporation,
The Postgres Database Company.
Attachment | Content-Type | Size |
---|---|---|
pg_stat_local_bk_j4.txt | text/plain | 51.2 KB |
pg_stat_local_normal_backup.txt | text/plain | 97.1 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Andy Fan | 2020-05-22 06:10:29 | Re: Planning counters in pg_stat_statements (using pgss_store) |
Previous Message | Andy Fan | 2020-05-22 06:02:52 | Re: Planning counters in pg_stat_statements (using pgss_store) |