From: | Laurenz Albe <laurenz(dot)albe(at)cybertec(dot)at> |
---|---|
To: | pgsql-hackers(at)lists(dot)postgresql(dot)org |
Subject: | Speeding up COPY TO for uuids and arrays |
Date: | 2024-02-17 16:48:23 |
Message-ID: | 679d5455cbbb0af667ccb753da51a475bae1eaed.camel@cybertec.at |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
While analyzing a customer's performance problem, I noticed that
the performance of pg_dump for large arrays is terrible.
As a test case, I created a table with 10000 rows, each of which
had an array of 10000 uuids. The table resided in shared buffers.
The following took 24.5 seconds:
COPY mytab TO '/dev/null';
Most of the time was spent in array_out and uuid_out.
I tried binary copy, which took 4.4 seconds:
COPY mytab TO '/dev/null' (FORMAT 'binary');
Here, a lot of time was spent in pq_begintypsend.
So I looked for low-hanging fruit, and the result is the attached
patch series.
- Patch 0001 speeds up pq_begintypsend with a custom macro.
That brought the binary copy down to 3.7 seconds, which is a
speed gain of 15%.
- Patch 0001 speeds up uuid_out by avoiding the overhead of
a Stringinfo. This brings text mode COPY to 19.4 seconds,
which is speed gain of 21%.
- Patch 0003 speeds up array_out a bit by avoiding some zero
byte writes. The measured speed gain is under 2%.
Yours,
Laurenz Albe
Attachment | Content-Type | Size |
---|---|---|
v1-0001-Speed-up-binary-COPY-TO.patch | text/x-patch | 2.0 KB |
v1-0002-Speed-up-uuid_out.patch | text/x-patch | 1.5 KB |
v1-0003-Small-speedup-for-array_out.patch | text/x-patch | 1.3 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Pavel Luzanov | 2024-02-17 18:06:16 | Re: Things I don't like about \du's "Attributes" column |
Previous Message | Francisco Olarte | 2024-02-17 15:12:15 | Re: BUG #18348: Inconsistency with EXTRACT([field] from INTERVAL); |