RE: [POC] Fast COPY FROM command for the table with foreign partitions

From: "Tang, Haiying" <tanghy(dot)fnst(at)cn(dot)fujitsu(dot)com>
To: Andrey Lepikhov <a(dot)lepikhov(at)postgrespro(dot)ru>
Cc: Alexey Kondratov <a(dot)kondratov(at)postgrespro(dot)ru>, Michael Paquier <michael(at)paquier(dot)xyz>, Ashutosh Bapat <ashutosh(dot)bapat(at)2ndquadrant(dot)com>, Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, Etsuro Fujita <etsuro(dot)fujita(at)gmail(dot)com>, PostgreSQL-Dev <pgsql-hackers(at)postgresql(dot)org>, "Hou, Zhijie" <houzj(dot)fnst(at)cn(dot)fujitsu(dot)com>, Amit Langote <amitlangote09(at)gmail(dot)com>, "Tsunakawa, Takayuki" <tsunakawa(dot)takay(at)fujitsu(dot)com>
Subject: RE: [POC] Fast COPY FROM command for the table with foreign partitions
Date: 2021-01-11 11:59:13
Message-ID: bc75078fc9614263a0d5f006dd6f0c3c@G08CNEXMBPEKD05.g08.fujitsu.local
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Andrey,

I had a general look at this extension feature, I think it's beneficial for some application scenarios of PostgreSQL. So I did 7 performance cases test on your patch(v13). The results are really good. As you can see below we can get 7-10 times improvement with this patch.

PSA test_copy_from.sql shows my test cases detail(I didn't attach my data file since it's too big).

Below are the test results:
'Test No' corresponds to the number(0 1...6) in attached test_copy_from.sql.
%reg=(Patched-Unpatched)/Unpatched), Unit is millisecond.

|Test No| Test Case |Patched(ms) | Unpatched(ms) |%reg |
|-------|-----------------------------------------------------------------------------------------|-------------|---------------|-------|
|0 |COPY FROM insertion into the partitioned table(parition is foreign table) | 102483.223 | 1083300.907 | -91% |
|1 |COPY FROM insertion into the partitioned table(parition is foreign partition) | 104779.893 | 1207320.287 | -91% |
|2 |COPY FROM insertion into the foreign table(without partition) | 100268.730 | 1077309.158 | -91% |
|3 |COPY FROM insertion into the partitioned table(part of foreign partitions) | 104110.620 | 1134781.855 | -91% |
|4 |COPY FROM insertion into the partitioned table with constraint(part of foreign partition)| 136356.201 | 1238539.603 | -89% |
|5 |COPY FROM insertion into the foreign table with constraint(without partition) | 136818.262 | 1189921.742 | -89% |
|6 |\copy insertion into the partitioned table with constraint. | 140368.072 | 1242689.924 | -89% |

If there is any question on my tests, please feel free to ask.

Best Regard,
Tang

Attachment Content-Type Size
test_copy_from.sql application/octet-stream 9.4 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Mark Zhao 2021-01-11 12:03:34 Re: adding partitioned tables to publications
Previous Message Bharath Rupireddy 2021-01-11 11:58:21 Re: Added schema level support for publication.