Re: Make COPY format extendable: Extract COPY TO format implementations

From: Sutou Kouhei <kou(at)clear-code(dot)com>
To: sawada(dot)mshk(at)gmail(dot)com
Cc: michael(at)paquier(dot)xyz, zhjwpku(at)gmail(dot)com, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Make COPY format extendable: Extract COPY TO format implementations
Date: 2025-03-01 02:50:09
Message-ID: 20250301.115009.424844407736647598.kou@clear-code.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

Our 0001/0002 patches were merged into master. I've rebased
on master. Can we discuss how to proceed rest patches?

The contents of them aren't changed but I'll show a summary
of them again:

0001-0003 are for COPY TO and 0004-0007 are for COPY FROM.

For COPY TO:

0001: Add support for adding custom COPY TO format. This
uses tablesample like handler approach. We've discussed
other approaches such as USING+CREATE XXX approach but it
seems that other approaches are overkill for this case.

See also: https://www.postgresql.org/message-id/flat/d838025aceeb19c9ff1db702fa55cabf%40postgrespro.ru#caca2799effc859f82f40ee8bec531d8

0002: Export CopyToStateData to implement custom COPY TO
format as extension.

0003: Export a function and add a private space to
CopyToStateData to implement custom COPY TO format as
extension.

We may want to squash 0002 and 0003 but splitting them will
be easy to review. Because 0002 just moves existing codes
(with some rename) and 0003 just adds some codes. If we
squash 0002 and 0003, moving and adding are mixed.

For COPY FROM:

0004: This is COPY FROM version of 0001.

0005: 0002 has COPY_ prefix -> COPY_DEST_ prefix change for
enum CopyDest. This is similar change for enum CopySource.

0006: This is COPY FROM version of 0003.

0007: This is for easy to implement "ON_ERROR stop" and
"LOG_VERBOSITY verbose" in extension.

We may want to squash 0005-0007 like for 0002-0003.

Thanks,
--
kou

Attachment Content-Type Size
v35-0001-Add-support-for-adding-custom-COPY-TO-format.patch text/x-patch 19.5 KB
v35-0002-Export-CopyToStateData-as-private-data.patch text/x-patch 9.7 KB
v35-0003-Add-support-for-implementing-custom-COPY-TO-form.patch text/x-patch 2.4 KB
v35-0004-Add-support-for-adding-custom-COPY-FROM-format.patch text/x-patch 9.2 KB
v35-0005-Use-COPY_SOURCE_-prefix-for-CopySource-enum-valu.patch text/x-patch 3.5 KB
v35-0006-Add-support-for-implementing-custom-COPY-FROM-fo.patch text/x-patch 2.4 KB
v35-0007-Add-CopyFromSkipErrorRow-for-custom-COPY-format-.patch text/x-patch 11.0 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Noah Misch 2025-03-01 03:50:31 Re: Licence preamble update
Previous Message wenhui qiu 2025-03-01 00:57:52 Re: Trigger more frequent autovacuums of heavy insert tables