Re: Loading 500m json files to database

From: pinker <pinker(at)onet(dot)eu>
To: pgsql-general(at)postgresql(dot)org
Subject: Re: Loading 500m json files to database
Date: 2020-03-24 00:22:02
Message-ID: 1585009322858-0.post@n3.nabble.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Ertan Küçükoğlu wrote
> However, if possible, you may think of using a local physical computer to
> do all uploading and after do backup/restore on cloud system.
>
> Compressed backup will be far less internet traffic compared to direct
> data inserts.

I was thinking about that but data source is a blob storage, so downloading
it first and then loading locally it's couple days extra for processing :/
it's not that fast even when I'm doing it locally ... so that would be like
extra 2 steps overhead :/

--
Sent from: https://www.postgresql-archive.org/PostgreSQL-general-f1843780.html

In response to

Browse pgsql-general by date

  From Date Subject
Next Message pinker 2020-03-24 00:23:14 Re: Loading 500m json files to database
Previous Message pinker 2020-03-24 00:18:45 Re: Loading 500m json files to database