From: | Tony Caduto <tony(dot)caduto(at)amsoftwaredesign(dot)com> |
---|---|
To: | pgsql-bugs(at)postgresql(dot)org |
Subject: | problem with 8.0rc1 not using indexes (more info) |
Date: | 2004-12-18 22:27:31 |
Message-ID: | 41C4AED3.6080904@amsoftwaredesign.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-bugs |
I have tracked the problem down to the aggregate COUNT function.
In the below code with the count function in the proc the execution time
per 1000 rows of import increases after each commit of 1000.
Please see these screen shots showing the importer running under 7.4.5
and 8.0 rc1
http://www.amsoftwaredesign.com/downloads/pg_screenshots/745_run.PNG
http://www.amsoftwaredesign.com/downloads/pg_screenshots/800rc1_run.PNG
you can see that on the 7.4.5 the times between remain constant, but on
the 8.0 they increase after each commit of 1000 rows.
here are the explains for each:
http://www.amsoftwaredesign.com/downloads/pg_screenshots/function_explain_745.PNG
http://www.amsoftwaredesign.com/downloads/pg_screenshots/function_explain_800rc1.PNG
I ended up commenting out sections of the function to see where the
problem is and form the below extract you can see what is happening.
The first select grabs the account ID for the account number passed in
via accountnumber_IN, the second select graps the security ID from the
cusip passed in.
The third select does a count against the positions table, and this is
the table that grows as the importer progresses through the raw file.
with the aggregate function in place is when the index for the select
appears not to be used. It's strange as when I do a explain on the
select by itself it does indicate it is doing a index scan.
It really seems to be a problem with count inside of the function.
My 8.0 server is exactly the same as the 7.4.5 (identicle compaq dl380s
wth 2gb of ram and 2..4gzh P4 processors. I simply did a dump of the
exact same database on the 7 box and restored it on the 8 box. I also
did a index rebuild and vacuum analyszed the 8.0 box and I continually
get the same results. The 7.4.5 box finishes the same import routine 25
minutes faster than the 8.0 box.
I won't be able to upgrade my production box until this is resolved.
I can post the full function if need be, just let me know.
SELECT account_id
FROM common.COMMON_ACCOUNT_DETAIL
WHERE account_number = accountnumber_IN AND
administrator <> 'CLD' AND
portfolio_manager <> 'CLD'
INTO varAccountID;
SELECT security_id
FROM common.COMMON_SECURITY
WHERE cusip = cusip_IN
INTO varSecurityID;
SELECT count(*)
FROM common.COMMON_POSITIONS
WHERE security_id = varSecurityID AND
account_id = varAccountID
INTO varPositionCount;
IF (varAccountID IS NOT NULL AND varPositionCount < 1 ) THEN
INSERT INTO common.COMMON_POSITIONS (
account_id,
security_id,
quantity,
cost,
market_value,
holding_percent )
VALUES (
varAccountID,
varSecurityID,
quantity_IN,
cost_IN,
varSecurityMV,
varHoldingPercent );
END IF;
Thanks
Tony Caduto
From | Date | Subject | |
---|---|---|---|
Next Message | Tony Caduto | 2004-12-19 03:37:44 | Re: Postgresql 8.0 RC1 not using indexes |
Previous Message | Charles Tse | 2004-12-18 20:39:55 | Jdbc3SimpleDataSource does not require the setting of password to access database |