Re: importing in sql

On Mon, Jul 10, 2000 at 05:03:08PM -0500, Abdul Karim wrote:
> Hi, I am trying to import loads of data into postgres, I am having
> trouble with a field which is a sequence. I know how to import data
> using a delimiter with the copy command. But I need to know how I
> increase the sequence on each line of import.
>
> My Table has the following format.
>
> field1 serial primary key,
> field2 char (50),
> field3 char(50),
> field4 int,
> field5 date
>
> The file has the following format
>
> field2|field3|field4|field5|
>
> Each field is separated by a | (pipe), How do I insert the sequence
> number before field2? I have like 30 files in this format and I need to
> import them in one table with each line having unique number. Has
> anyone come across similar situation? any help would be greatly
> appreciated.
>

you don't mention your operating system, but on Linux I'd do this
with awk:

awk '{print NR"|"$0}' data_file > data_file.out

You mention 30 files: to get them sequential, I'd do:

cat files1 file2 [...] | awk '{print NR"|"$0}' | split -C 1m - dataout

Use some glob pattern for the 'cat' command that puts the files in the
order you want (even if it's just listing them all individually)

Awk will then number them, and split will generate file with at most 1
Meg of lines in them. If you're not worried about doing an all in one
bulk load, skip the split.

After this is all loaded, be sure to set the sequence associated with
the serial filed you loading into:

SELECT setval('table_field_seq',max(field)) from table;

Ross
--
Ross J. Reedstrom, Ph.D., <reedstrm(at)rice(dot)edu>
NSBRI Research Scientist/Programmer
Computer and Information Technology Institute
Rice University, 6100 S. Main St., Houston, TX 77005