From: | David Rowley <david(dot)rowley(at)2ndquadrant(dot)com> |
---|---|
To: | Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com> |
Cc: | PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, khuddleston(at)pivotal(dot)io, Melanie Plageman <melanieplageman(at)gmail(dot)com> |
Subject: | Re: Making "COPY partitioned_table FROM" faster |
Date: | 2018-07-24 04:27:59 |
Message-ID: | CAKJS1f8kJJQLMLmboUCr6th7WdB-0eB_BuyLDUQ1gt6f+uCDcA@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On 24 July 2018 at 06:38, Peter Eisentraut
<peter(dot)eisentraut(at)2ndquadrant(dot)com> wrote:
> On 20.07.18 16:57, David Rowley wrote:
>> One final note: I'm not entirely convinced we need this adaptive
>> code, but it seems easy enough to rip it back out if it's more trouble
>> than it's worth. But if the other option is a GUC, then I'd rather
>> stick with the adaptive code, it's likely going to do much better than
>> a GUC since it can change itself during the copy which will be useful
>> when the stream contains a mix small and large sets of consecutive
>> tuples which belong to the same partition.
>
> I think some kind of way to switch between the two code paths would be
> desirable. For example, with hash partitioning, it's likely that in
> many cases you won't find any adjacent candidates in batches of
> significant size. So then you've just made everything 5% slower.
> Unless we can make the multi-insert path itself faster.
>
> The particular heuristic you have chosen seems sensible to me, but we'll
> have to see how it holds up in practice.
Thanks for having a look at this. You're probably right here,
although, the slowdown I measured was 2.2%, not 5%. I had it in my
head that it was about 1%, but for 2.2% it seems worth the extra code.
I've attached a small delta against the v2 patch that fixes up a few
comments and gets rid of a needless assignment.
--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachment | Content-Type | Size |
---|---|---|
v2_copy_speedup_fixup_delta.patch | application/octet-stream | 1.9 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Michael Paquier | 2018-07-24 04:50:10 | Re: BUG #15182: Canceling authentication due to timeout aka Denial of Service Attack |
Previous Message | Andres Freund | 2018-07-24 04:17:53 | Re: BUG #15182: Canceling authentication due to timeout aka Denial of Service Attack |