Re: Index creation running now for 14 hours

From: pgsql-performance-owner(at)postgresql(dot)org [mailto:pgsql-performance-owner(at)postgresql(dot)org] On Behalf Of Tory M Blue
Sent: Wednesday, August 26, 2015 3:14 PM
To: pgsql-performance <pgsql-performance(at)postgresql(dot)org>
Subject: [PERFORM] Index creation running now for 14 hours

I'm running 9.3.4 with slon 2.2.3, I did a drop add last night at 9pm, it started this particular tables index creation at 10:16pm and it's still running. 1 single core is at 100% (32 core box) and there is almost zero I/O activity.

CentOS 6.6

16398 | clsdb | 25765 | 10 | postgres | slon.remoteWorkerThread_1 | 10.13.200.232 | | 45712 | 2015-08-25 21:12:01.6
19819-07 | 2015-08-25 21:22:08.68766-07 | 2015-08-25 22:16:03.10099-07 | 2015-08-25 22:16:03.100992-07 | f | active | select "_cls".fini
shTableAfterCopy(143); analyze "torque"."impressions";
I was wondering if there were underlying tools to see how it's progressing, or if there is anything I can do to bump the performance mid creation? Nothing I can do really without stopping postgres or slon, but that would start me back at square one.

Thanks
Tory

iostat: sdb is the db directory

avg-cpu: %user %nice %system %iowait %steal %idle
3.55 0.00 0.23 0.00 0.00 96.22

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 1.00 0.00 12.00 0 24
sdb 0.00 0.00 0.00 0 0

avg-cpu: %user %nice %system %iowait %steal %idle
3.57 0.00 0.06 0.00 0.00 96.37

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.00 0.00 0.00 0 0
sdb 21.50 0.00 15484.00 0 30968

avg-cpu: %user %nice %system %iowait %steal %idle
3.72 0.00 0.06 0.00 0.00 96.22

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 2.00 0.00 20.00 0 40
sdb 0.00 0.00 0.00 0 0

avg-cpu: %user %nice %system %iowait %steal %idle
4.06 0.00 0.05 0.02 0.00 95.87

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 4.00 0.00 64.00 0 128
sdb 3.50 0.00 108.00 0 216

avg-cpu: %user %nice %system %iowait %steal %idle
3.36 0.00 0.03 0.00 0.00 96.61

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.00 0.00 0.00 0 0
sdb 0.00 0.00 0.00 0 0

avg-cpu: %user %nice %system %iowait %steal %idle
3.41 0.00 0.06 0.00 0.00 96.53

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.00 0.00 0.00 0 0
sdb 0.00 0.00 0.00 0 0

avg-cpu: %user %nice %system %iowait %steal %idle
3.45 0.00 0.27 0.00 0.00 96.28

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 0.00 0.00 0.00 0 0
sdb 1.00 0.00 24.00 0 48

avg-cpu: %user %nice %system %iowait %steal %idle
3.50 0.00 0.30 0.00 0.00 96.20

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 1.50 0.00 344.00 0 688
sdb 0.00 0.00 0.00 0 0

Check pg_locks in regards to the table in question.

Regards,
Igor Neyman