Re: Loading 500m json files to database

From: Adrian Klaver <adrian(dot)klaver(at)aklaver(dot)com>
To: pinker <pinker(at)onet(dot)eu>, pgsql-general(at)postgresql(dot)org
Subject: Re: Loading 500m json files to database
Date: 2020-03-24 00:50:46
Message-ID: ea5aa03e-0801-3864-78e1-ed8a56daccb9@aklaver.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

On 3/23/20 5:23 PM, pinker wrote:
> it's a cloud and no plpythonu extension avaiable unfortunately

I presume Python itself is available, so would it not be possible to
create a program that concatenates the files into batches and COPY(s)
that data into Postgres using the Psycopg2 COPY functions:

https://www.psycopg.org/docs/usage.html#using-copy-to-and-copy-from

>
>
>
> --
> Sent from: https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html
>
>

--
Adrian Klaver
adrian(dot)klaver(at)aklaver(dot)com

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message David G. Johnston 2020-03-24 01:11:28 Re: Loading 500m json files to database
Previous Message Adrian Klaver 2020-03-24 00:44:56 Re: Loading 500m json files to database