Re: Adapter update.

From: Richard Huxton <dev(at)archonet(dot)com>
To: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
Cc: Murali Maddali <murali(dot)maddali(at)uai(dot)com>, pgsql-general(at)postgresql(dot)org
Subject: Re: Adapter update.
Date: 2007-08-23 17:48:20
Message-ID: 46CDC864.8050102@archonet.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Joshua D. Drake wrote:
>>
>> I have added transaction to my code and it took about 2 and half hours to
>> process around 48,000 records. Again all this time is taken by update method
>> on the adapter.
>>
>> I don't know Perl to setup the database link to SQL Server 2005 and also I
>> don't have permission to write the data to files. Are there any other
>> options like a different driver I can use or through stored procedures. I
>> have to compare each column in each row before doing the update.
>
> This is probably where your time is spent, not the actual commit of the
> data. 48k records is nothing.

Ditto what Joshua says. Loading that many records should take minutes
not hours.

Try this last bit of my first reply.

>> Load the data into an import table (TEMPORARY table probably) and then just
>> use three queries to handle deletion, update and insertion.
>> Comparing one row at a time is adding a lot of overhead.

--
Richard Huxton
Archonet Ltd

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Erik Jones 2007-08-23 17:48:37 Re: Apache + PHP + Postgres Interaction
Previous Message Joshua D. Drake 2007-08-23 17:42:34 Re: Adapter update.