COPY from STDIN vs file with large CSVs

From: Wells Oliver <wells(dot)oliver(at)gmail(dot)com>
To: pgsql-admin <pgsql-admin(at)postgresql(dot)org>
Subject: COPY from STDIN vs file with large CSVs
Date: 2020-01-08 16:54:14
Message-ID: CAOC+FBVjk4W=nhn3FZUiTgroFFhaQ6bvh-NiU207PhD9nO3Qww@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

I have a CSV that's ~30GB. Some 400m rows. Would there be a meaningful
performance difference to run COPY from STDIN using: cat f.csv | psql "COPY
.. FROM STDIN WITH CSV" versus just doing "COPY ... FROM 'f.csv' WITH CSV"?

Thanks. It took about four hours to copy one and I felt that was a little
much.

--
Wells Oliver
wells(dot)oliver(at)gmail(dot)com <wellsoliver(at)gmail(dot)com>

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Indirani Venkopa Chetty 2020-01-08 17:05:17 RE: COPY from STDIN vs file with large CSVs
Previous Message Shrikant Bhende 2020-01-08 05:23:18 Re: Alter command getting stuck on a table