RE: COPY performance on Windows

From: "Ryohei Takahashi (Fujitsu)" <r(dot)takahashi_2(at)fujitsu(dot)com>
To: "Ryohei Takahashi (Fujitsu)" <r(dot)takahashi_2(at)fujitsu(dot)com>, 'Robert Haas' <robertmhaas(at)gmail(dot)com>
Cc: "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: RE: COPY performance on Windows
Date: 2024-12-11 12:18:23
Message-ID: TY3PR01MB11891C0FD066F069B113A2376823E2@TY3PR01MB11891.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

I continuously investigate the performance problem of COPY on Windows.

I noticed that not only PG17.0 but also PG16.6 have performance problem compared to PG16.4.
The performance is 2.5%-5.8% worse, especially when the number of clients is 1 or 2.

I modified the performance measurement script of the thread in [1].
* Enabled to run on Windows git bash
* Enabled to compare PG16.4, PG16.6 and PG17.0
* Increase the row number to 10 times (about 10GB)

I measured on Windows Server 2022 machine with 44 core CPU and 512GB memory.
The results are following.

* PG16.4
PG164: nclients = 1, time = 432
PG164: nclients = 2, time = 238
PG164: nclients = 4, time = 157
PG164: nclients = 8, time = 135
PG164: nclients = 16, time = 163
PG164: nclients = 32, time = 261
PG164: nclients = 64, time = 458
PG164: nclients = 128, time = 611
PG164: nclients = 256, time = 622

* PG16.6
PG166: nclients = 1, time = 444 (2.7% worse than PG16.4)
PG166: nclients = 2, time = 252 (5.8% worse than PG16.4)
PG166: nclients = 4, time = 156
PG166: nclients = 8, time = 135
PG166: nclients = 16, time = 163
PG166: nclients = 32, time = 261
PG166: nclients = 64, time = 458
PG166: nclients = 128, time = 612
PG166: nclients = 256, time = 621

* PG17.0
PG170: nclients = 1, time = 448 (3.7% worse than PG16.4)
PG170: nclients = 2, time = 244 (2.5% worse than PG16.4)
PG170: nclients = 4, time = 159
PG170: nclients = 8, time = 137
PG170: nclients = 16, time = 165
PG170: nclients = 32, time = 262
PG170: nclients = 64, time = 458
PG170: nclients = 128, time = 611
PG170: nclients = 256, time = 621

(1)
I attach the performance measurement script.
If you have a Windows environment, could you please reproduce the same performance problem?

(2)
The performance of PG16.6 and PG17.0 are worse than PG16.4.
So, I think the commits between August and September affects the performance.
I will analyze these commits.

[1] https://postgr.es/m/CAD21AoDvDmUQeJtZrau1ovnT_smN940=Kp6mszNGK3bq9yRN6g@mail.gmail.com

Regards,
Ryohei Takahashi

Attachment Content-Type Size
test.sh application/octet-stream 2.6 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Guillaume Lelarge 2024-12-11 12:41:09 Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE
Previous Message Jakub Wartak 2024-12-11 12:05:21 Re: FileFallocate misbehaving on XFS