Re: pgsql: Refactor COPY FROM to use format callback functions.

On Fri, Feb 28, 2025 at 11:47 AM Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:
>
>
> On 2025-02-28 Fr 1:31 PM, Masahiko Sawada wrote:
>
> Refactor COPY FROM to use format callback functions.
>
> This commit introduces a new CopyFromRoutine struct, which is a set of
> callback routines to read tuples in a specific format. It also makes
> COPY FROM with the existing formats (text, CSV, and binary) utilize
> these format callbacks.
>
> This change is a preliminary step towards making the COPY FROM command
> extensible in terms of input formats.
>
> Similar to 2e4127b6d2d, this refactoring contributes to a performance
> improvement by reducing the number of "if" branches that need to be
> checked on a per-row basis when sending field representations in text
> or CSV mode. The performance benchmark results showed ~5% performance
> gain in text or CSV mode.
>
> Author: Sutou Kouhei <kou(at)clear-code(dot)com>
> Reviewed-by: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>
> Reviewed-by: Michael Paquier <michael(at)paquier(dot)xyz>
> Reviewed-by: Andres Freund <andres(at)anarazel(dot)de>
> Reviewed-by: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>
> Reviewed-by: Junwang Zhao <zhjwpku(at)gmail(dot)com>
> Discussion: https://postgr.es/m/20231204.153548.2126325458835528809.kou@clear-code.com
>
>
>
> This patch has completely broken the file_textarray fdw, which uses NextCopyFromRawFields(). Removing that from API is not a good thing.
>

Thank you for pointing it out.

I've just posted my analysis[1] and am planning to revive that API
(Sutou-san already proposed an idea). Could you please check if the
idea would work for file_text_array_fdw?

Regards,

[1] https://www.postgresql.org/message-id/CAD21AoDr13%3Ddx%2Bk8gmQnR5_bY%2BNskyN4mbSWN0KhQncL6xuPMA%40mail.gmail.com

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com