Re: Multiple index builds on same table - in one sweep?

From: Shaun Thomas <sthomas(at)peak6(dot)com>
To: Chris Ruprecht <chris(at)ruprecht(dot)org>
Cc: <pgsql-performance(at)postgresql(dot)org>
Subject: Re: Multiple index builds on same table - in one sweep?
Date: 2011-04-11 15:13:41
Message-ID: 4DA31AA5.20101@peak6.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

On 04/09/2011 11:28 AM, Chris Ruprecht wrote:

> I'm wondering if there is a way to build these indexes in parallel
> while reading the table only once for all indexes and building them
> all at the same time. Is there an index build tool that I missed
> somehow, that can do this?

I threw together a very crude duo of shell scripts to do this. I've
attached them for you. To use them, you make a file named tablist.txt
which contains the names of all the tables you want to reindex, and then
you run them like this:

bash generate_rebuild_scripts.sh my_database 8
bash launch_rebuild_scripts.sh my_database

The first one in the above example would connect to my_database and
create eight scripts that would run in parallel, with indexes ordered
smallest to largest to prevent one script from getting stuck with
several large indexes while the rest got small ones. The second script
just launches them and makes a log directory so you can watch the progress.

I've run this with up to 16 concurrent threads without major issue. It
comes in handy.

--
Shaun Thomas
OptionsHouse | 141 W. Jackson Blvd. | Suite 800 | Chicago IL, 60604
312-676-8870
sthomas(at)peak6(dot)com

______________________________________________

See http://www.peak6.com/email_disclaimer.php
for terms and conditions related to this email

Attachment Content-Type Size
generate_rebuild_scripts.sh application/x-sh 1.1 KB
launch_rebuild_scripts.sh application/x-sh 181 bytes

In response to

Browse pgsql-performance by date

  From Date Subject
Next Message Anne Rosset 2011-04-11 16:07:45 Re: Slow query postgres 8.3
Previous Message Saurabh Agrawal 2011-04-11 14:10:55 Postgres 9 slave lagging