Re: Partition performance causing ddl commands to slow down significantly

From: <fburgess(at)radiantblue(dot)com>
To: "Andres Freund" <andres(at)2ndquadrant(dot)com>
Cc: pgsql-bugs(at)postgresql(dot)org
Subject: Re: Partition performance causing ddl commands to slow down significantly
Date: 2013-04-12 18:46:30
Message-ID: 20130412114630.5a830134ae84016b0174832fdc1a3173.fc7948457c.wbe@email11.secureserver.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

<html><body><span style="font-family:Verdana; color:#000000; font-size:10pt;"><div>Sorry, Is this visible? </div>
<div>&nbsp;</div>
<div>We are having performance related problems on one of our big data Partition tables. The table is partitioned by date and the partitions are organized from Jan 2003 thru Dec 2013. <BR>We have 268 child partitions associated with the Parent table, and we have constraint_exclusion=partition set. </div>
<div>The execution of the SQL query:&nbsp; select count(*) from dna_strands; </div>
<div>yields:&nbsp; QUERY PLAN<BR>_____________________________________________________________________________________________&nbsp;&nbsp; </div>
<div>Aggregate (cost=2246778.49..2246778.50 rows=1 width=0)<BR>&nbsp; -&gt; Append (0.00..2159647.04 rows=34852580 width=0)<BR>&nbsp;&nbsp;&nbsp;&nbsp; -&gt; Seq Scan on dna_strands (cost=0.00..0.00 rows=1 width)<BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Filter: (cid = 1)<BR>&nbsp;&nbsp;&nbsp;&nbsp; -&gt; Index Scan using dna_strands_y2003m01_cid on dna_strands_y2003m01 dna_strands (cost=0.00..677652 rows=1 width=0)<BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Index Cond: (cid = 1)<BR>&nbsp;&nbsp;&nbsp;&nbsp; -&gt; Index Scan using dna_strands_y2003m02_cid on dna_strands_y2003m02 dna_strands (cost=0.00..974423 rows=1 width=0)<BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Index Cond: (cid = 1)<BR>&nbsp;&nbsp;&nbsp;&nbsp; -&gt; Index Scan using dna_strands_y2003m03_cid on dna_strands_y2003m03 dna_strands (cost=0.00..992301 rows=1 width=0)<BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Index Cond: (cid = 1)<BR>&nbsp;&nbsp;&nbsp;&nbsp; ...<BR>&nbsp;&nbsp;&nbsp;&nbsp; ...</div>
<div>&nbsp;&nbsp;&nbsp;&nbsp; -&gt; Index Scan using dna_strands_y2013m12_cid on dna_strands_y2013m12 dna_strands (cost=0.00..8.27 rows=1 width=0)<BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Index Cond: (cid = 1)</div>
<div>Question: Is there any way to modify the Planner to do the inverse of the Index Scan's.&nbsp; In other words, to start the index scans in reverse order from <BR>the most recent date to the oldest date, i.e. "dna_strands_y2013m12" backwards. Our application users query much more heavily at the most recent data that <BR>has been ingested into the PostgreSQL database.&nbsp; Would this capability speed up query performance?</div>
<div>&nbsp;</div>
<div>Thanks</div>
<div><BR>&nbsp;</div>
<div>&nbsp;</div>
<BLOCKQUOTE style="BORDER-LEFT: blue 2px solid; PADDING-LEFT: 8px; FONT-FAMILY: verdana; COLOR: black; MARGIN-LEFT: 8px; FONT-SIZE: 10pt" id=replyBlockquote webmail="1">
<DIV id=wmQuoteWrapper>-------- Original Message --------<BR>Subject: Re: [BUGS] Partition performance causing ddl commands to slow<BR>down significantly<BR>From: Andres Freund &lt;<a href="mailto:andres(at)2ndquadrant(dot)com">andres(at)2ndquadrant(dot)com</a>&gt;<BR>Date: Fri, April 12, 2013 11:36 am<BR>To: <a href="mailto:fburgess(at)radiantblue(dot)com">fburgess(at)radiantblue(dot)com</a><BR><BR>Hi,<BR><BR>On 2013-04-12 11:31:33 -0700, <a href="mailto:fburgess(at)radiantblue(dot)com">fburgess(at)radiantblue(dot)com</a> wrote:<BR>&gt; &lt;html&gt;&lt;body&gt;&lt;span style="font-family:Verdana; color:#000000; font-size:10pt;"&gt;&lt;div&gt;We are having performance related problems on one of our big data Partition tables. The table is partitioned by date and the partitions are organized from Jan 2003 thru Dec 2013. &lt;BR&gt;We have 268 child partitions associated with the Parent table, and we have constraint_exclusion=partition set. &lt;/div&gt;<BR>&gt; &lt;div&gt;&amp;nbsp;&lt;/div&gt;<BR>&gt; &lt;div&gt;The execution of the SQL query:&amp;nbsp; select count(*) from dna_strands; &lt;/div&gt;<BR>&gt; &lt;div&gt;yields:&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; QUERY PLAN&lt;BR&gt;_____________________________________________________________________________________________&amp;nbsp;&amp;nbsp; &lt;/div&gt;<BR>&gt; &lt;div&gt;Aggregate (cost=2246778.49..2246778.50 rows=1 width=0)&lt;BR&gt;&amp;nbsp; -&amp;gt; Append (0.00..2159647.04 rows=34852580 width=0)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; -&amp;gt; Seq Scan on dna_strands (cost=0.00..0.00 rows=1 width)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; Filter: (cid = 1)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; -&amp;gt; Index Scan using dna_strands_y2003m01_cid on dna_strands_y2003m01 dna_strands (cost=0.00..677652 rows=1 width=0)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; Index Cond: (cid = 1)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; -&amp;gt; Index Scan using dna_strands_y2003m02_cid on dna_strands_y2003m02 dna_strands (cost=0.00..974423 rows=1 width=0)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; Index Cond: (cid = 1)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; -&amp;gt; Index Scan using dna_strands_y2003m03_cid on dna_strands_y2003m03 dna_strands (cost=0.00..992301 rows=1 width=0)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; Index Cond: (cid = 1)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; ...&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; ...&lt;/div&gt;<BR>&gt; &lt;div&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; ...&lt;/div&gt;<BR>&gt; &lt;div&gt;&amp;nbsp;&lt;/div&gt;<BR>&gt; &lt;div&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; -&amp;gt; Index Scan using dna_strands_y2013m12_cid on dna_strands_y2013m12 dna_strands (cost=0.00..8.27 rows=1 width=0)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; Index Cond: (cid = 1)&lt;/div&gt;<BR>&gt; &lt;div&gt;Question: Is there any way to modify the Planner to do the inverse of the Index Scan's.&amp;nbsp; In other words, to start the index scans in reverse order from &lt;BR&gt;the most recent date to the oldest date, i.e. "dna_strands_y2013m12" backwards. Our application users query much more heavily at the most recent data that &lt;BR&gt;has been ingested into the PostgreSQL database.&amp;nbsp; Would this capability speed up query performance?&lt;/div&gt;<BR>&gt; &lt;div&gt;Thanks&lt;/div&gt;<BR>&gt; &lt;div&gt;&lt;BR&gt;&amp;nbsp;&lt;/div&gt;&lt;/span&gt;&lt;/body&gt;&lt;/html&gt;<BR><BR>Youre sending completely unreadable html only mails again.<BR><BR>Greetings,<BR><BR>Andres Freund<BR><BR>-- <BR>Andres Freund <a href="http://www.2ndQuadrant.com">http://www.2ndQuadrant.com</a>/<BR>PostgreSQL Development, 24x7 Support, Training &amp; Services<BR></DIV></BLOCKQUOTE></span></body></html>

Attachment Content-Type Size
unknown_filename text/html 7.1 KB

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message John R Pierce 2013-04-12 18:53:11 Re: Partition performance causing ddl commands to slow down significantly
Previous Message fburgess 2013-04-12 18:31:33 Partition performance causing ddl commands to slow down significantly