optimizer cost calculation problem

From: Tatsuo Ishii <t-ishii(at)sra(dot)co(dot)jp>
To: pgsql-hackers(at)postgresql(dot)org
Subject: optimizer cost calculation problem
Date: 2003-03-31 22:37:26
Message-ID: 20030401.073726.112626637.t-ishii@sra.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Kenji Sugita has identified a problem with cost_sort() in costsize.c.

In the following code fragment, sortmembytes is defined as long. So

double nruns = nbytes / (sortmembytes * 2);

may cause an integer overflow if sortmembytes exceeds 2^30, which in
turn make optimizer to produce wrong query plan(this actually happned
in a large PostgreSQL installation which has tons of memory). Here is a
proposed fix against current:

*** optimizer/path/costsize.c 16 Feb 2003 02:30:38 -0000 1.107
--- optimizer/path/costsize.c 31 Mar 2003 22:25:06 -0000
***************
*** 490,496 ****
Cost startup_cost = input_cost;
Cost run_cost = 0;
double nbytes = relation_byte_size(tuples, width);
! long sortmembytes = SortMem * 1024L;

if (!enable_sort)
startup_cost += disable_cost;
--- 490,496 ----
Cost startup_cost = input_cost;
Cost run_cost = 0;
double nbytes = relation_byte_size(tuples, width);
! double sortmembytes = SortMem * 1024.0;

if (!enable_sort)
startup_cost += disable_cost;

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Ed L. 2003-03-31 22:38:12 Re: index corruption?
Previous Message Bruce Momjian 2003-03-31 20:53:55 Re: ltree patch