Re: Inserting 26 million rows takes 8 hours, how to improve those times?

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Jose Vicente Nunez Z <josevnz(at)newbreak(dot)com>
Cc: pgsql-admin(at)postgresql(dot)org
Subject: Re: Inserting 26 million rows takes 8 hours, how to improve those times?
Date: 2003-09-16 14:44:36
Message-ID: 17211.1063723476@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

Jose Vicente Nunez Z <josevnz(at)newbreak(dot)com> writes:
> For that i'm using the PostgreSQL copy tool, but the problem is that is
> taking almost 9 hours to load the data:

Hmm, 889 rows/second doesn't sound that bad, considering that you've
given us no data about the hardware you're running on. In particular,
what's your disk setup?

But at any rate, the only bottlenecks I could think of for a COPY
command are:

1. Data conversion. This would depend on the column datatypes of the
table, which you didn't tell us.

2. Index updating. This would depend on the indexes on the table,
which you didn't tell us.

3. Foreign-key constraint checking. This would depend on the foreign
keys the table has, which you didn't tell us.

In short ... you'd better provide a lot more detail if you want
useful commentary.

> I apologize if this not the proper place to post this questions.

pgsql-performance would be more appropriate, probably.

regards, tom lane

In response to

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Jose Vicente Nunez Z 2003-09-16 15:27:46 Re: Inserting 26 million rows takes 8 hours, how to
Previous Message Thierry Missimilly 2003-09-16 13:51:37 Re: postgresql headers