Re: Logging parallel worker draught

On 2/27/24 15:09, Tomas Vondra wrote> That is certainly true, but it's
not a new thing, I believe. IIRC we may
> not report statistics until the end of the transaction, so no progress
> updates, I'm not sure what happens if the doesn't end correctly (e.g.
> backend dies, ...). Similarly for the temporary files, we don't report
> those until the temporary file gets closed, so I'm not sure you'd get a
> lot of info about the progress either.
>
> I admit I haven't tried and maybe I don't remember the details wrong.
> Might be useful to experiment with this first a little bit ...

Ah, yes, Tempfile usage is reported on tempfile closure or deletion
for shared file sets.

> I certainly understand that, and I realize it may feel daunting and even
> discouraging. What I can promise is that I'm willing to help, either by
> suggesting ways to approach the problems, doing reviews, and so on.
> Would that be sufficient for you to continue working on this patch?

Yes, thanks for the proposal, I'll work on it on report here. I am otherwise
booked on company projects so I cannot promise a quick progress.

> Some random thoughts/ideas about how to approach this:
>
> - For parallel workers I think this might be as simple as adding some
> counters into QueryDesc, and update those during query exec (instead of
> just logging stuff directly). I'm not sure if it should be added to
> "Instrumentation" or separately.

I will start here to see how it works.

--
Benoit Lobréau
Consultant
http://dalibo.com