Re: Issue with copying data from a text file.

Hi,

I think I had the exact same problem as you do a while back and I
solved it by removing the header row and the "CSV HEADER" clause of the
statement. For the large files I had, it was easier (for me) to remove
the header row than it was to escape out all the quotes (or regen the
file):

COPY deal_lines_temp_load FROM
'c:/temp/autodrs_deal_lines.txt'
WITH DELIMITER AS '^';

I think the parser doesn't look for nor generate quoted rows except
when CSV is specified.. It would be nice if there was a way to specify
a "HEADER" row without invoking CSV parsing rules (friendly hint to
core devs!) :)

Let us all know if that works!

Steve

At 03:14 AM 3/20/2007, you wrote:
>Date: Tue, 20 Mar 2007 11:25:38 +0900
>From: Paul Lambert <paul(dot)lambert(at)autoledgers(dot)com(dot)au>
>To: pgsql-sql(at)postgresql(dot)org
>Subject: Issue with copying data from a text file.
>Message-ID: <45FF4622(dot)2010404(at)autoledgers(dot)com(dot)au>
>
>I have a procedure in place that copies data from a caret delimited
>text
>file into a table storing some information.
>
>One of the fields in the table contains an item description which may
>contain item dimensions such as - 17" alloy wheels
>
>The problem I am getting when I do my load is I believe due to the
>presence of the double quotation marks giving the copy the impression
>that it is to include the information following as a single text
>string
>until it gets to the next set of double quotes. As a result, I get the
>
>following:
>
>AutoDRS=# COPY deal_lines_temp_load FROM
>'c:/temp/autodrs_deal_lines.txt'
>WITH DELIMITER AS '^' CSV HEADER;
>ERROR: value too long for type character varying(30)
>CONTEXT: COPY deal_lines_temp_load, line 87, column order_desc: "17 5
>
>spoke alloy wheels.^1291.18^117.38^983.69^1291.18^^C^^
>
>The column as you can see is defined as a 30 character field, the load
>
>contains in this column ^17" 5 spoke alloy wheels.^
>
>I note an option in the COPY command to specify the quote character,
>defaulting to double quote. The problem being a single quote will also
>
>be used in the data, as will other characters. Is there any way to get
>a
>copy to have no quote character? I.e. read the file and put whatever
>is
>between the caret characters straight into the appropriate field
>exactly
>as is.
>
>TIA,
>Paul.
>
>--
>Paul Lambert
>Database Administrator
>AutoLedgers