From: | Andres Freund <andres(at)anarazel(dot)de> |
---|---|
To: | Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> |
Cc: | David Rowley <dgrowleyml(at)gmail(dot)com>, Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>, Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>, Andrew Dunstan <andrew(at)dunslane(dot)net>, Peter Eisentraut <peter(at)eisentraut(dot)org>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
Subject: | Re: Performance degradation on concurrent COPY into a single relation in PG16. |
Date: | 2023-08-07 18:10:26 |
Message-ID: | 20230807181026.encacqh7w7svsqiy@awork3.anarazel.de |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hi,
On 2023-08-07 23:05:39 +0900, Masahiko Sawada wrote:
> On Mon, Aug 7, 2023 at 3:16 PM David Rowley <dgrowleyml(at)gmail(dot)com> wrote:
> >
> > On Wed, 2 Aug 2023 at 13:35, David Rowley <dgrowleyml(at)gmail(dot)com> wrote:
> > > So, it looks like this item can be closed off. I'll hold off from
> > > doing that for a few days just in case anyone else wants to give
> > > feedback or test themselves.
> >
> > Alright, closed.
>
> IIUC the problem with multiple concurrent COPY is not resolved yet.
Yea - it was just hard to analyze until the other regressions were fixed.
> The result of nclients = 1 became better thanks to recent fixes, but
> there still seems to be the performance regression at nclient = 2~16
> (on RHEL 8 and 9). Andres reported[1] that after changing
> MAX_BUFFERED_TUPLES to 5000 the numbers became a lot better but it
> would not be the solution, as he mentioned.
I think there could be a quite simple fix: Track by how much we've extended
the relation previously in the same bistate. If we already extended by many
blocks, it's very likey that we'll do so further.
A simple prototype patch attached. The results for me are promising. I copied
a smaller file [1], to have more accurate throughput results at shorter runs
(15s).
HEAD before:
clients tps
1 41
2 76
4 136
8 248
16 360
32 375
64 317
HEAD after:
clients tps
1 43
2 80
4 155
8 280
16 369
32 405
64 344
Any chance you could your benchmark? I don't see as much of a regression vs 16
as you...
Greetings,
Andres Freund
[1] COPY (SELECT generate_series(1, 100000)) TO '/tmp/data.copy';
Attachment | Content-Type | Size |
---|---|---|
remember_extension_size.diff | text/x-diff | 1.7 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2023-08-07 18:24:58 | Re: Using defines for protocol characters |
Previous Message | Robert Haas | 2023-08-07 17:36:43 | Re: Using defines for protocol characters |