Re: Report checkpoint progress in server logs

> I think the right choice to solve the *general* problem is the
> mentioned pg_stat_progress_checkpoints.
>
> We may want to *additionally* have the ability to log the progress
> specifically for the special cases when we're not able to use that
> view. And in those case, we can perhaps just use the existing
> log_startup_progress_interval parameter for this as well -- at least
> for the startup checkpoint.

+1

> We need at least a trace of the number of buffers to sync (num_to_scan) before the checkpoint start, instead of just emitting the stats at the end.
>
> Bharat, it would be good to show the buffers synced counter and the total buffers to sync, checkpointer pid, substep it is running, whether it is on target for completion, checkpoint_Reason
> (manual/times/forced). BufferSync has several variables tracking the sync progress locally, and we may need some refactoring here.

I agree to provide above mentioned information as part of showing the
progress of current checkpoint operation. I am currently looking into
the code to know if any other information can be added.

Thanks & Regards,
Nitin Jadhav

On Thu, Jan 6, 2022 at 5:12 AM Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>
> On Wed, Dec 29, 2021 at 10:40:59AM -0500, Tom Lane wrote:
> > Magnus Hagander <magnus(at)hagander(dot)net> writes:
> > >> Therefore, reporting the checkpoint progress in the server logs, much
> > >> like [1], seems to be the best way IMO.
> >
> > > I find progress reporting in the logfile to generally be a terrible
> > > way of doing things, and the fact that we do it for the startup
> > > process is/should be only because we have no other choice, not because
> > > it's the right choice.
> >
> > I'm already pretty seriously unhappy about the log-spamming effects of
> > 64da07c41 (default to log_checkpoints=on), and am willing to lay a side
> > bet that that gets reverted after we have some field experience with it.
> > This proposal seems far worse from that standpoint. Keep in mind that
> > our out-of-the-box logging configuration still doesn't have any log
> > rotation ability, which means that the noisier the server is in normal
> > operation, the sooner you fill your disk.
>
> I think we are looking at three potential observable behaviors people
> might care about:
>
> * the current activity/progress of checkpoints
> * the historical reporting of checkpoint completion, mixed in with other
> log messages for later analysis
> * the aggregate behavior of checkpoint operation
>
> I think it is clear that checkpoint progress activity isn't useful for
> the server logs because that information has little historical value,
> but does fit for a progress view. As Tom already expressed, we will
> have to wait to see if non-progress checkpoint information in the logs
> has sufficient historical value.
>
> --
> Bruce Momjian <bruce(at)momjian(dot)us> https://momjian.us
> EDB https://enterprisedb.com
>
> If only the physical world exists, free will is an illusion.
>
>
>