COPY from .csv File and Remove Duplicates

From: Rich Shepard <rshepard(at)appl-ecosys(dot)com>
To: pgsql-general(at)postgresql(dot)org
Subject: COPY from .csv File and Remove Duplicates
Date: 2011-08-11 23:13:29
Message-ID: alpine.LNX.2.00.1108111607310.14240@salmo.appl-ecosys.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

A table has a sequence to generate a primary key for inserted records with
NULLs in that column.

I have a .csv file of approximately 10k rows to copy into this table. My
two questions which have not been answered by reference to my postgres
reference book or Google searches are:

1) Will the sequence automatically add the nextval() to each new record as
the copy command runs?

2) Many of these rows almost certainly are already in the table. I would
like to remove duplicates either during the COPY command or immediately
after. I'm considering copying the new data into a clone of the table then
running a SELECT to add only those rows in the new cloned table to the
existing table.

Suggestions on getting these data in without duplicating existing rows
will be very helpful.

TIA,

Rich

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Rich Shepard 2011-08-11 23:33:25 Re: suggestions for archive_command to a remote standby
Previous Message Naoko Reeves 2011-08-11 23:09:22 Re: Regex Query Index question