Re: Make COPY format extendable: Extract COPY TO format implementations

On Fri, Dec 8, 2023 at 3:27 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>
> On Fri, Dec 8, 2023 at 1:39 AM Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:
> >
> >
> > On 2023-12-07 Th 03:37, Junwang Zhao wrote:
> > >
> > > The point of this refactor (from my view) is to make it possible to add new
> > > copy handlers in extensions, just like access method. As Andres suggested,
> > > a system catalog like *pg_copy_handler*, if we split TO and FROM into two
> > > sets of routines, does that mean we have to create two catalog(
> > > pg_copy_from_handler and pg_copy_to_handler)?
> >
> >
> >
> > Surely not. Either have two fields, one for the TO handler and one for
> > the FROM handler, or a flag on each row indicating if it's a FROM or TO
> > handler.

If we wrap the two fields into a single structure, that will still be in
copy.h, which I think is not necessary. A single routing wrapper should
be enough, the actual implementation still stays separate
copy_[to/from].c files.

>
> True.
>
> But why do we need a system catalog like pg_copy_handler in the first
> place? I imagined that an extension can define a handler function
> returning a set of callbacks and the parser can lookup the handler
> function by name, like FDW and TABLESAMPLE.
>
I can see FDW related utility commands but no TABLESAMPLE related,
and there is a pg_foreign_data_wrapper system catalog which has
a *fdwhandler* field.

If we want extensions to create a new copy handler, I think
something like pg_copy_hander should be necessary.

> Regards,
>
> --
> Masahiko Sawada
> Amazon Web Services: https://aws.amazon.com

I go one step further to implement the pg_copy_handler, attached V5 is
the implementation with some changes suggested by Kou.

You can also review this on this github pull request [1].

[1]: https://github.com/zhjwpku/postgres/pull/1/files

--
Regards
Junwang Zhao