Re: Online checksums verification in the backend

On Mon, Dec 9, 2019 at 5:21 PM Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>
> On Fri, Dec 6, 2019 at 9:51 AM Julien Rouhaud <rjuju123(at)gmail(dot)com> wrote:
>
> > This brings the second consideration: how to report the list corrupted
> > blocks to end users. As I said this is for now returned via the SRF,
> > but this is clearly not ideal and should rather be made available more
> > globally.
>
> Some people might prefer notices, because you can get those while the
> thing is still running, rather than a result set, which you will only
> see when the query finishes. Other people might prefer an SRF, because
> they want to have the data in structured form so that they can
> postprocess it. Not sure what you mean by "more globally."

I meant having the results available system-wide, not only to the
caller. I think that emitting a log/notice level should always be
done on top on whatever other communication facility we're using.

> I guess one
> idea would be to provide a way to kick this off in the background via
> a background worker or similar and then have it put the results in a
> table. But that might fail if there are checksum errors in the
> catalogs themselves.

Yes that's a concern. We could maintain a list in (dynamic) shared
memory with a simple SQL wrapper to read the data, but that would be
lost with a crash/restart. Or use
pgstat_report_checksum_failures_in_db(), modifying it to get an
relfilenode, bocknum and forknum and append that to some flat files,
hoping that it won't get corrupted either.