Re: Emitting JSON to file using COPY TO

From: "Euler Taveira" <euler(at)eulerto(dot)com>
To: "Daniel Verite" <daniel(at)manitou-mail(dot)org>, "Andrew Dunstan" <andrew(at)dunslane(dot)net>
Cc: "Joe Conway" <mail(at)joeconway(dot)com>, "Davin Shearer" <davin(at)apache(dot)org>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Emitting JSON to file using COPY TO
Date: 2023-12-07 02:42:06
Message-ID: 6ab82c2f-c19f-4cd6-a2a9-ddf9832e6209@app.fastmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

On Wed, Dec 6, 2023, at 3:59 PM, Daniel Verite wrote:
> The first Copy data message with contents "5b0a" does not qualify
> as a row of data with 3 columns as advertised in the CopyOut
> message. Isn't that a problem?
>
> At least the json non-ARRAY case ("json lines") doesn't have
> this issue, since every CopyData message corresponds effectively
> to a row in the table.

Moreover, if your interface wants to process the COPY data stream while
receiving it, you cannot provide "json array" format because each row (plus all
of the received ones) is not a valid JSON. Hence, a JSON parser cannot be
executed until you receive the whole data set. (wal2json format 1 has this
disadvantage. Format 2 was born to provide a better alternative -- each row is
a valid JSON.) I'm not saying that "json array" is not useful but that for
large data sets, it is less useful.

--
Euler Taveira
EDB https://www.enterprisedb.com/

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Nathan Bossart 2023-12-07 02:56:22 Re: Emitting JSON to file using COPY TO
Previous Message David G. Johnston 2023-12-07 01:21:28 Re: Emitting JSON to file using COPY TO

Browse pgsql-hackers by date

  From Date Subject
Next Message Nathan Bossart 2023-12-07 02:56:22 Re: Emitting JSON to file using COPY TO
Previous Message Michael Paquier 2023-12-07 02:40:30 Re: Is WAL_DEBUG related code still relevant today?