Re: Emitting JSON to file using COPY TO

From: "Daniel Verite" <daniel(at)manitou-mail(dot)org>
To: "Joe Conway" <mail(at)joeconway(dot)com>
Cc: Andrew Dunstan <andrew(at)dunslane(dot)net>, Davin Shearer <davin(at)apache(dot)org>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Emitting JSON to file using COPY TO
Date: 2023-12-07 13:35:52
Message-ID: 3df907b2-0a3a-4597-8ef1-707b553667cd@manitou-mail.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

Joe Conway wrote:

> The attached should fix the CopyOut response to say one column. I.e. it
> ought to look something like:

Spending more time with the doc I came to the opinion that in this bit
of the protocol, in CopyOutResponse (B)
...
Int16
The number of columns in the data to be copied (denoted N below).
...

this number must be the number of columns in the source.
That is for COPY table(a,b,c) the number is 3, independently
on whether the result is formatted in text, cvs, json or binary.

I think that changing it for json can reasonably be interpreted
as a protocol break and we should not do it.

The fact that this value does not help parsing the CopyData
messages that come next is not a new issue. A reader that
doesn't know the field separator and whether it's text or csv
cannot parse these messages into fields anyway.
But just knowing how much columns there are in the original
data might be useful by itself and we don't want to break that.

The other question for me is, in the CopyData message, this
bit:
" Messages sent from the backend will always correspond to single data rows"

ISTM that considering that the "[" starting the json array is a
"data row" is a stretch.
That might be interpreted as a protocol break, depending
on how strict the interpretation is.

Best regards,
--
Daniel Vérité
https://postgresql.verite.pro/
Twitter: @DanielVerite

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message David G. Johnston 2023-12-07 13:47:10 Re: Emitting JSON to file using COPY TO
Previous Message Andrew Dunstan 2023-12-07 13:19:19 Re: Emitting JSON to file using COPY TO

Browse pgsql-hackers by date

  From Date Subject
Next Message Sacha Hottinger 2023-12-07 13:43:55 AW: Building PosgresSQL with LLVM fails on Solaris 11.4
Previous Message Jeevan Chalke 2023-12-07 13:24:31 Re: More new SQL/JSON item methods