From: | Andres Freund <andres(at)anarazel(dot)de> |
---|---|
To: | Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> |
Cc: | David Rowley <dgrowleyml(at)gmail(dot)com>, Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>, Jakub Wartak <jakub(dot)wartak(at)enterprisedb(dot)com>, Andrew Dunstan <andrew(at)dunslane(dot)net>, Peter Eisentraut <peter(at)eisentraut(dot)org>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
Subject: | Re: Performance degradation on concurrent COPY into a single relation in PG16. |
Date: | 2023-08-12 20:05:04 |
Message-ID: | 20230812200504.li5vomg36j2tretx@awork3.anarazel.de |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hi,
On 2023-08-08 12:45:05 +0900, Masahiko Sawada wrote:
> > I think there could be a quite simple fix: Track by how much we've extended
> > the relation previously in the same bistate. If we already extended by many
> > blocks, it's very likey that we'll do so further.
> >
> > A simple prototype patch attached. The results for me are promising. I copied
> > a smaller file [1], to have more accurate throughput results at shorter runs
> > (15s).
>
> Thank you for the patch!
Attached is a mildly updated version of that patch. No functional changes,
just polished comments and added a commit message.
> > Any chance you could your benchmark? I don't see as much of a regression vs 16
> > as you...
>
> Sure. The results are promising for me too:
>
> nclients = 1, execution time = 13.743
> nclients = 2, execution time = 7.552
> nclients = 4, execution time = 4.758
> nclients = 8, execution time = 3.035
> nclients = 16, execution time = 2.172
> nclients = 32, execution time = 1.959
> nclients = 64, execution time = 1.819
> nclients = 128, execution time = 1.583
> nclients = 256, execution time = 1.631
Nice. We are consistently better than both 15 and "post integer parsing 16".
I'm really a bit baffled at myself for not using this approach from the get
go.
This also would make it much more beneficial to use a BulkInsertState in
nodeModifyTable.c, even without converting to table_multi_insert().
I was tempted to optimize RelationAddBlocks() a bit, by not calling
RelationExtensionLockWaiterCount() if we are already extending by
MAX_BUFFERS_TO_EXTEND_BY. Before this patch, it was pretty much impossible to
reach that case, because of the MAX_BUFFERED_* limits in copyfrom.c, but now
it's more common. But that probably ought to be done only HEAD, not 16.
A related oddity: RelationExtensionLockWaiterCount()->LockWaiterCount() uses
an exclusive lock on the lock partition - which seems not at all necessary?
Unless somebody sees a reason not to, I'm planning to push this?
Greetings,
Andres Freund
Attachment | Content-Type | Size |
---|---|---|
v2-0001-hio-Take-number-of-prior-relation-extensions-into.patch | text/x-diff | 4.6 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Andres Freund | 2023-08-12 21:00:06 | Re: A failure in 031_recovery_conflict.pl on Debian/s390x |
Previous Message | Andres Freund | 2023-08-12 19:12:07 | Re: Fix pg_stat_reset_single_table_counters function |