Re: Should CSV parsing be stricter about mid-field quotes?

From: "Joel Jacobson" <joel(at)compiler(dot)org>
To: "Daniel Verite" <daniel(at)manitou-mail(dot)org>
Cc: "Andrew Dunstan" <andrew(at)dunslane(dot)net>, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: Should CSV parsing be stricter about mid-field quotes?
Date: 2023-05-20 07:16:30
Message-ID: 79f0b67a-c526-4f5b-a7cf-9bafea2f4ee4@app.fastmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, May 19, 2023, at 18:06, Daniel Verite wrote:
> COPY FROM file CSV somewhat differs as your example shows,
> but it still mishandle \. when unquoted. For instance, consider this
> file to load with COPY t FROM '/tmp/t.csv' WITH CSV
> $ cat /tmp/t.csv
> line 1
> \.
> line 3
> line 4
>
> It results in having only "line 1" being imported.

Hmm, this is a problem for one of the new use-cases I brought up that would be
possible with DELIMITER NONE QUOTE NONE, i.e. to import unstructured log files,
where each raw line should be imported "as is" into a single text column.

Is there a valid reason why \. is needed for COPY FROM filename?
It seems to me it would only be necessary for the COPY FROM STDIN case,
since files have a natural end-of-file and a known file size.

/Joel

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Drouvot, Bertrand 2023-05-20 08:37:58 Re: PG 16 draft release notes ready
Previous Message Joel Jacobson 2023-05-20 06:59:46 Re: New COPY options: DELIMITER NONE and QUOTE NONE