Re: [WIP] Patches to enable extraction state of query execution from external session

> Hi,
>
> On 2016-08-29 18:22:56 +0300, maksim wrote:
>> Now I complete extension that provides facility to see the current state of
>> query execution working on external session in form of EXPLAIN ANALYZE
>> output. This extension works on 9.5 version, for 9.6 and later it doesn't
>> support detailed statistics for parallel nodes yet.
>
> Could you expand a bit on what you want this for exactly?

Max goal - to push my extension to postgres core. But now it's ready
only for 9.5. Prerequisites of this extension are patches presented here.

>> 2. Patch that enables to interrupt the query executor
>> (executor_hooks.patch).
>> This patch enables to hang up hooks on executor function of each node
>> (ExecProcNode). I define hooks before any node execution and after
>> execution.
>> I use this patch to add possibility of query tracing by emitted rows from
>> any node. I interrupt query executor after any node delivers one or zero
>> rows to upper node. And after execution of specific number trace steps I can
>> get the query state of traceable backend which will be somewhat
>> deterministic. I use this possibility for regression tests of my extension.
>
> This will increase executor overhead.

In simple case we have checks on existence of hooks. We may suppose to
use only not heavy processing on hooks under regular execution of query.
In my case (query trace), I set up hooks under trace mode and throw off
otherwise.

> I think we'll need to find a way
> to hide this behind the existing if (instrument) branches.

And so can be. It doesn't matter for trace mode. But I think instrument
branch is intended only for collecting statistics by nodes.

>> 3. Patch that enables to output runtime explain statistics
>> (runtime_explain.patch).
>> This patch extends the regular explain functionality. The problem is in the
>> point that regular explain call makes result output after query execution
>> performing InstrEndLoop on nodes where necessary. My patch introduces
>> specific flag *runtime* that indicates whether we explain running query and
>> does some insertions in source code dedicated to output the statistics of
>> running query.
>
> Unless I'm missing something this doesn't really expose a user of this
> functionality?

This patch is only for extension. As described in
https://www.postgresql.org/message-id/CACACo5Sz7G0MFauC082iM=XX_hQ7qQ5ndR4JPo+H-O5vp6iCcQ@mail.gmail.com
regular ExplainNode() prints only statistics after query execution and
asserts under InstEndLoop(). My patch releases this problem and rewrite
formulas for statistic parameters appropriate to running queries without
affecting regular EXPLAIN outputs.

--
Maksim Milyutin
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company