Re: Multiple COPY on the same table

From: Nicolas Paris <nicolas(dot)paris(at)riseup(dot)net>
To: pgsql-general(at)lists(dot)postgresql(dot)org
Subject: Re: Multiple COPY on the same table
Date: 2018-08-20 15:56:01
Message-ID: 20180820155601.kkjsvbt7hr7esyag@riseup.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

> Can I split a large file into multiple files and then run copy using
> each file.

AFAIK, copy command locks the table[1] while there is no mention of this
in the documentation[2].

> Will the performance boost by close to 4x??

You might be interested in the pbBulkInsert tool[3] that allows parallel
copy with some succes accordingly to benchmarks. However, that tool does
not handle multiline csv. Because of that limitation I have been using
the standard copy command with binary format with some succes.

[1] https://grokbase.com/t/postgresql/pgsql-general/01597pv3qs/copy-locking
[2] https://www.postgresql.org/docs/current/static/sql-copy.html
[3] https://github.com/bytefish/PgBulkInsert

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Ken Tanzer 2018-08-20 16:07:35 Re: Re: Re: Allow Reg Expressions in Position function
Previous Message Tom Lane 2018-08-20 15:29:22 Re: help defining a basic type operator