Re: [HACKERS] CLUSTER command progress monitor

From: Tattsu Yama <yamatattsu(at)gmail(dot)com>
To: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>
Cc: Tatsuro Yamada <tatsuro(dot)yamada(dot)tf(at)nttcom(dot)co(dot)jp>, robertmhaas(at)gmail(dot)com, Rafia Sabih <rafia(dot)pghackers(at)gmail(dot)com>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, Peter Geoghegan <pg(at)bowt(dot)ie>, pgsql-hackers(at)postgresql(dot)org, Jeff Janes <jeff(dot)janes(at)gmail(dot)com>, Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>, Michael Paquier <michael(dot)paquier(at)gmail(dot)com>, Tattsu Yama <yamatattsu(at)gmail(dot)com>
Subject: Re: [HACKERS] CLUSTER command progress monitor
Date: 2019-09-14 04:06:32
Message-ID: CAOKkKFs9e0_dbH+pVFnQEZ6epvYQKdY0sakrYbwMHNMbB8i3Kw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Alvaro!

> Hello Tatsuro,
> On 2019-Aug-13, Tatsuro Yamada wrote:
> > On 2019/08/02 3:43, Alvaro Herrera wrote:
> > > Hmm, I'm trying this out now and I don't see the index_rebuild_count
> > > ever go up. I think it's because the indexes are built using parallel
> > > index build ... or maybe it was the table AM changes that moved things
> > > around, not sure. There's a period at the end when the CLUSTER command
> > > keeps working, but it's gone from pg_stat_progress_cluster.
> >
> > Thanks for your report.
> > I'll investigate it. :)
>

I have fixed it. Can you please verify?
>

Thanks! I can review your patch for fix it.
However, I was starting fixing the problem from the last day of PGConf.Asia
(11 Sep).
Attached file is WIP patch.In my patch, I added "command id" to all APIs of
progress reporting to isolate commands. Therefore, it doesn't allow to
cascade updating system views. And my patch is on WIP so it needs clean-up
and test.
I share it anyway. :)

Here is a test result of my patch.
The last column index_rebuild count is increased.
========================================
postgres=# select * from pg_stat_progress_cluster ; \watch 0.001;
11636|13591|postgres|16384|CLUSTER|initializing|0|0|0|0|0|0
11636|13591|postgres|16384|CLUSTER|index scanning heap|16389|251|251|0|0|0
...
11636|13591|postgres|16384|CLUSTER|index scanning
heap|16389|10000|10000|0|0|0
11636|13591|postgres|16384|CLUSTER|rebuilding
index|16389|10000|10000|0|0|0...
11636|13591|postgres|16384|CLUSTER|rebuilding index|16389|10000|10000|0|0|1
...
11636|13591|postgres|16384|CLUSTER|rebuilding index|16389|10000|10000|0|0|2
...
11636|13591|postgres|16384|CLUSTER|rebuilding index|16389|10000|10000|0|0|3
...
11636|13591|postgres|16384|CLUSTER|rebuilding index|16389|10000|10000|0|0|4
...
11636|13591|postgres|16384|CLUSTER|performing final
cleanup|16389|10000|10000|0|0|5
========================================

Thanks,
Tatsuro Yamada

Attachment Content-Type Size
v1_fix_progress_report_for_cluster.patch application/octet-stream 38.3 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2019-09-14 04:20:06 Re: pg_rewind docs correction
Previous Message Amit Kapila 2019-09-14 03:55:31 Re: [HACKERS] [PATCH] pageinspect function to decode infomasks