Re: pg_waldump stucks with options --follow or -f and --stats or -z

On Tue, Nov 16, 2021 at 8:26 AM Michael Paquier <michael(at)paquier(dot)xyz> wrote:
>
> On Mon, Nov 15, 2021 at 07:32:38PM +0530, Bharath Rupireddy wrote:
> > pg_waldump options, --follow or -f(to keep polling once per second for
> > new WAL to appear) and --stats or -z don't work well together i.e. the
> > command stucks [1]. I think the pg_waldump should emit an error. Note
> > that the pg_basebakup does error out on incompatible options.
> >
> > Here's a small patch for fixing above along with a note in the documentation.
> >
> > Thoughts?
>
> I don't think that we should block this combination of options as you
> are proposing. The existing behavior is useful for users when it
> comes to an end position specified with -e, to be able to gather some
> stats on a cluster or an archive where we may not have all the
> contents wanted yet.

You are right. The "./pg_waldump -p data/ -s 0/14CC090 -e 0/14CC390 -f
-z" would fail with what I proposed which isn't a good thing at all.
Should we block both the combinations "-s/-f/-z", "-s/-e/-f/-z"?

> > [1] The following commands stuck:
> > ./pg_waldump -p data/ -s 0/7000060 -f -z
> > ./pg_waldump -p data/ -s 0/7000060 -f --stats='record'
> > ./pg_waldump -p data/ -s 0/7000060 -f --stats='rmgr'
>
> Saying that, you are not completely wrong either, as following
> something while we won't print any stats at all is not really
> helpful. Another thing I can think of here is to make pg_waldump
> more responsive to the dump of the stats when interrupted, via
> XLogDumpDisplayStats().

It looks okay to be more responsive with the -f option so that the
above commands keep emitting stats for every 1 second(the --follow
option waits for 1 second). Should we really be emitting stats for
every 1 second? Isn't there going to be too much information on the
stdout? Or should we be emitting the stats for every 5 or 10 seconds?

If we be more responsive and keep emitting stats per 1 or 5 or 10 etc.
seconds(we can give an option to the user to choose when he/she would
like to see the stats with -f option, but this is going to be an
overkill IMO), can the user make anything out of those loads of stats
getting emitted? Another idea is to give some incremental stats but it
is overkill again IMO. And when the pg_waldump has the capability to
emit the stats, why to block it?

In summary, we have the following options:
1) block the combinations "-s/-f/-z", "-s/-e/-f/-z"
2) be more responsive and keep emitting the stats per 1 sec default
with -f option
3) be more responsive and keep emitting the stats per user's choice
of seconds (a new option that can be used with the -s/-e/-f/-z).

Thoughts?

Regards,
Bharath Rupireddy.