Re: Should CSV parsing be stricter about mid-field quotes?

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Andrew Dunstan <andrew(at)dunslane(dot)net>
Cc: Joel Jacobson <joel(at)compiler(dot)org>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Should CSV parsing be stricter about mid-field quotes?
Date: 2023-05-13 13:45:41
Message-ID: 1728940.1683985541@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> I could see an argument for a STRICT mode which would disallow partially
> quoted fields, although I'd like some evidence that we're dealing with a
> real problem here. Is there really a CSV producer that produces output
> like that you showed in your example? And if so has anyone objected to
> them about the insanity of that?

I think you'd want not just "some evidence" but "compelling evidence".
Any such option is going to add cycles into the low-level input parser
for COPY, which we know is a hot spot and we've expended plenty of
sweat on. Adding a speed penalty that will be paid by the 99.99%
of users who don't have an issue here is going to be a hard sell.

regards, tom lane

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Ron 2023-05-13 18:28:11 Re: Adding SHOW CREATE TABLE
Previous Message Andrew Dunstan 2023-05-13 12:44:48 Re: Should CSV parsing be stricter about mid-field quotes?