Problem using COPY command to load data

From: Glen Beane <Glen(dot)Beane(at)jax(dot)org>
To: "pgsql-general(at)postgresql(dot)org" <pgsql-general(at)postgresql(dot)org>
Subject: Problem using COPY command to load data
Date: 2008-11-11 19:03:32
Message-ID: C53F4134.1FC6%glen.beane@jax.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

I'm not sure if this is the write place to post this question or not, but I
hope someone can help me out.

I am using the copy_from command from the python psycopg2 library to do some
bulk data loading of a postgres database. This had been working OK, until
my script barfed because I was being careless, and it seemed to leave the
database in a strange state that I can't recover from. Here is what
happened:

I am connecting to postgres via TCP, but I happen to be running the script
on the same host ast the server right now. I had forgotten to clear the
contents of one table before loading it from a file and the copy_from
command exited with error because of conflicting primary keys. After this
any time I attempted to execute copy_from it would just hang (after
successfully connecting to the database and doing a series of DELETE FROM
SQL commands).

I just see something like this if I look through the output of ps:

postgres 30432 1.7 1.2 42884 12512 ? Ss 13:16 0:00 postgres:
biomart mgi_biomart_intermediate 127.0.1.1(41831) COPY

The script hangs forever, and I don't see anything in the postgres log.

If I switched my script to connect through a unix socket (remove
host='hostname' from the psycopg2.connect() call), then it works again.
I've restarted postgresql many times, the TCP connection always hangs on the
COPY from my script now.

I have fixed my script to be more careful and handle the copy_from() error
in a more sane way, but I'd like to fix whatever is now wrong with postgres.

Here are some entries from pg_hba.conf:

host all all 10.0.0.0/8 md5
host all all 127.0.0.1/32 md5
host all all 127.0.1.1/32 md5
local all biomart md5

Everything else in pg_hba.conf is default
The only change to postgresql.conf is to change the listen_address to '*'

Any insight would be appreciated.

--
Glen L. Beane
Software Engineer
The Jackson Laboratory
Phone (207) 288-6153

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Alvaro Herrera 2008-11-11 19:12:26 Re: SHMMAX and shared_bufffers
Previous Message Scott Marlowe 2008-11-11 18:53:55 Re: SHMMAX and shared_bufffers