From: | Thomas Schmidt <postgres(at)stephan(dot)homeunix(dot)net> |
---|---|
To: | pgsql-general(at)postgresql(dot)org |
Subject: | pgloader an Indexes / was: Re: CSV-bulk import and defaults |
Date: | 2011-01-03 08:48:22 |
Message-ID: | 4D218D56.6080405@stephan.homeunix.net |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
Hello,
Am 03.01.11 00:06, schrieb Adrian Klaver:
> On Sunday 02 January 2011 2:22:14 pm Thomas Schmidt wrote:
>> well, I'm new to postgres and this is my post on this list :-)
>> Anyway, I've to batch-import bulk-csv data into a staging database (as
>> part of an ETL-"like" pocess). The data ought to be read via STDIN,
>> however for keeping in simple and stupid, saving it to a file and
>> importing afterwards is also an option. Sticking my nose into the docs,
>> I noticed that copy[1] as well as pg_import[2] are able to do it.
>>
>> However, there are some additional columns of the staging table (job id,
>> etc.) that have to be set in order to identify imported rows. These
>> attributes are not part of the data coming from STDIN (since its
>> meta-data) and I see no way for specifying default values for "missing"
>> cvs columns. (imho copy and pg_bulkload will use table defaults for
>> missing rows - do I miss something?).
>>
>> [1] http://www.postgresql.org/docs/9.0/static/sql-copy.html
>> [2] http://pgbulkload.projects.postgresql.org/pg_bulkload.html
> Check out pgloader:
> http://pgloader.projects.postgresql.org/
Thanks a lot - that's what I need. :-)
Btw. What about indexes?
http://www.postgresql.org/docs/9.0/interactive/populate.html suggests to
remove indexes before importing via copy (for obvious reasons).
Does pgloader take indexes into account or do I need to handle 'em manually?
Thanks in adance,
Thomas
From | Date | Subject | |
---|---|---|---|
Next Message | Andre Lopes | 2011-01-03 11:11:56 | Need advise for database structure for non linear data. |
Previous Message | Alban Hertroys | 2011-01-03 07:24:03 | Re: problem updating from form |