From: | Richard Huxton <dev(at)archonet(dot)com> |
---|---|
To: | "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com> |
Cc: | Murali Maddali <murali(dot)maddali(at)uai(dot)com>, pgsql-general(at)postgresql(dot)org |
Subject: | Re: Adapter update. |
Date: | 2007-08-23 17:48:20 |
Message-ID: | 46CDC864.8050102@archonet.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
Joshua D. Drake wrote:
>>
>> I have added transaction to my code and it took about 2 and half hours to
>> process around 48,000 records. Again all this time is taken by update method
>> on the adapter.
>>
>> I don't know Perl to setup the database link to SQL Server 2005 and also I
>> don't have permission to write the data to files. Are there any other
>> options like a different driver I can use or through stored procedures. I
>> have to compare each column in each row before doing the update.
>
> This is probably where your time is spent, not the actual commit of the
> data. 48k records is nothing.
Ditto what Joshua says. Loading that many records should take minutes
not hours.
Try this last bit of my first reply.
>> Load the data into an import table (TEMPORARY table probably) and then just
>> use three queries to handle deletion, update and insertion.
>> Comparing one row at a time is adding a lot of overhead.
--
Richard Huxton
Archonet Ltd
From | Date | Subject | |
---|---|---|---|
Next Message | Erik Jones | 2007-08-23 17:48:37 | Re: Apache + PHP + Postgres Interaction |
Previous Message | Joshua D. Drake | 2007-08-23 17:42:34 | Re: Adapter update. |