Speeding up COPY TO for uuids and arrays

From: Laurenz Albe <laurenz(dot)albe(at)cybertec(dot)at>
To: pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Speeding up COPY TO for uuids and arrays
Date: 2024-02-17 16:48:23
Message-ID: 679d5455cbbb0af667ccb753da51a475bae1eaed.camel@cybertec.at
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

While analyzing a customer's performance problem, I noticed that
the performance of pg_dump for large arrays is terrible.

As a test case, I created a table with 10000 rows, each of which
had an array of 10000 uuids. The table resided in shared buffers.

The following took 24.5 seconds:

COPY mytab TO '/dev/null';

Most of the time was spent in array_out and uuid_out.

I tried binary copy, which took 4.4 seconds:

COPY mytab TO '/dev/null' (FORMAT 'binary');

Here, a lot of time was spent in pq_begintypsend.

So I looked for low-hanging fruit, and the result is the attached
patch series.

- Patch 0001 speeds up pq_begintypsend with a custom macro.
That brought the binary copy down to 3.7 seconds, which is a
speed gain of 15%.

- Patch 0001 speeds up uuid_out by avoiding the overhead of
a Stringinfo. This brings text mode COPY to 19.4 seconds,
which is speed gain of 21%.

- Patch 0003 speeds up array_out a bit by avoiding some zero
byte writes. The measured speed gain is under 2%.

Yours,
Laurenz Albe

Attachment Content-Type Size
v1-0001-Speed-up-binary-COPY-TO.patch text/x-patch 2.0 KB
v1-0002-Speed-up-uuid_out.patch text/x-patch 1.5 KB
v1-0003-Small-speedup-for-array_out.patch text/x-patch 1.3 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Pavel Luzanov 2024-02-17 18:06:16 Re: Things I don't like about \du's "Attributes" column
Previous Message Francisco Olarte 2024-02-17 15:12:15 Re: BUG #18348: Inconsistency with EXTRACT([field] from INTERVAL);