Re: New "raw" COPY format

From: "Joel Jacobson" <joel(at)compiler(dot)org>
To: "Daniel Verite" <daniel(at)manitou-mail(dot)org>
Cc: "Jacob Champion" <jacob(dot)champion(at)enterprisedb(dot)com>, "Tatsuo Ishii" <ishii(at)postgresql(dot)org>, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: New "raw" COPY format
Date: 2024-10-16 19:13:55
Message-ID: 12aac0fe-5d5c-43ea-b530-e9fbe8160651@app.fastmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Oct 16, 2024, at 20:30, Joel Jacobson wrote:
> A final thought is to maybe consider just skipping
> the automagical newline detection for RAW?
>
> Instead of the automagical detection,
> the default newline delimiter could be the OS default,
> similar to how COPY TO works.
>
> That way, it would almost always just work for most users,
> as long as processing files within their OS,
> and when not, they would just need to specify the DELIMITER.

I would guess that nowadays, dealing with unstructured text files
are probably less common, than dealing with structured text files,
such as JSON, YAML, TOML, XML, etc.

Therefore, maybe DELIMITER NONE would be a better default
for RAW? Especially since it's then also more honest in being "raw".

If needing to import an unstructured text file that is just newline
delimited, and not wanting the entire file as a single value,
the newline style would then just need to be specified
using the DELIMITER option.

/Joel

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Masahiko Sawada 2024-10-16 19:25:40 Re: Using per-transaction memory contexts for storing decoded tuples
Previous Message Peter Geoghegan 2024-10-16 18:52:04 Re: Limiting overshoot in nbtree's parallel SAOP index scans