Re: New "raw" COPY format

From: jian he <jian(dot)universality(at)gmail(dot)com>
To: Joel Jacobson <joel(at)compiler(dot)org>
Cc: pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: New "raw" COPY format
Date: 2024-10-12 00:48:56
Message-ID: CACJufxFt15StJcJ-4SgJJauOUB9dr=sCVWY=Xd2BmriNHS4K2g@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Sat, Oct 12, 2024 at 5:02 AM Joel Jacobson <joel(at)compiler(dot)org> wrote:
>
> On Fri, Oct 11, 2024, at 22:29, Joel Jacobson wrote:
> > Hi hackers,
> >
> > This thread is about implementing a new "raw" COPY format.
> ...
> > The attached patch implements the above ideas.
> >
> > I think with these changes, it would be easier to hack on new and existing
> > copy options and formats.
> >
> > /Joel
> >
> > Attachments:
> > * v1-0001-Replace-binary-flags-binary-and-csv_mode-with-format.patch
> > * v1-0002-Reorganize-ProcessCopyOptions-for-clarity-and-consis.patch
>
> Ops, I see I failed to use the correct way to check if
> opts_out->force_notnull or opts_out->force_null
> have been set, that is using != NIL.
>
> However, thanks to not just blindly copy/pasting this code,
> I see I actually fixed a bug in HEAD, by also checking
> opts_out->force_notnull_all or opts_out->force_null_all,
> which HEAD currently fails to do:

git version 2.34.1
cannot do `git apply`

trying:
patch -p1 < patch -p1 <
$PATCHES/v2-0001-Replace-binary-flags-binary-and-csv_mode-with-format.patch
patch -p1 < $PATCHES/v2-0002-Reorganize-ProcessCopyOptions-for-clarity-and-consis.patch
After that, I still cannot apply.

typedef enum CopyFormat
{
COPY_FORMAT_TEXT,
COPY_FORMAT_BINARY,
COPY_FORMAT_CSV
} CopyFormat;
the last element should add a comma.

CopyFormat should add to
src/tools/pgindent/typedefs.list

+ /* Assert options have been set (defaults applied if not specified) */
+ Assert(opts_out->delim);
+ Assert(opts_out->null_print);
+
+ /* Don't allow the delimiter to appear in the NULL or DEFAULT
strings */
+
+ if (strchr(opts_out->null_print, opts_out->delim[0]) != NULL)

+ Assert(opts_out->delim);
+ Assert(opts_out->quote);
+ Assert(opts_out->null_print);
+
+ if (opts_out->delim[0] == opts_out->quote[0])
these Asserts, no need? Without it, if conditions are not met, it will
still segfault.

there is no sql example, like
copy the_table from :'filename' (format raw);

in the patch.
I thought you were going to implement something like that.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Joel Jacobson 2024-10-12 05:44:39 Re: [BUG FIX] Fix validation of COPY options FORCE_NOT_NULL/FORCE_NULL
Previous Message Masahiko Sawada 2024-10-12 00:43:04 Re: Missing deconstruct_array_builtin usage