Re: The flinfo->fn_extra question, from me this time.

On 21 Jul 2019, at 22:54, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
> Chapman Flack <chap(at)anastigmatix(dot)net> writes:
>> Until now, I had assumed that SFRM_ValuePerCall mode might offer some
>> benefits, such as the possibility of pipelining certain queries and not
>> building up a whole tuplestore in advance.
>
>> But looking in the code, I'm getting the impression that those
>> benefits are only theoretical future ones, as ExecMakeTableFunctionResult
>> implements SFRM_ValuePerCall mode by ... repeatedly calling the function
>> to build up a whole tuplestore in advance.
>
> Yes, that's the case for a SRF in FROM. A SRF in the targetlist
> actually does get the chance to pipeline, if it implements ValuePerCall.
>
> The FROM case could be improved perhaps, if somebody wanted to put
> time into it.

While looking at whether REFCURSOR output could be pipelined into the executor [1], I’ve stumbled upon the same.

By any chance, do either of you know if there are initiatives to make the changes mentioned?

> You'd still need to be prepared to build a tuplestore,
> in case of rescan or backwards fetch; but […]

I’m also interested in your comment here. If the function was STABLE, could not the function scan simply be restarted? (Rather than needing to create the tuplestore for all cases.)

I guess perhaps the backwards scan is where it falls down though...

> […] in principle you could return
> rows immediately while stashing them aside in a tuplestore.

Does the planner have any view on this? When I first saw what was going on, I presumed the planner had decided the cost of multiple function scans was greater than the cost of materialising it in a temporary store.

It occurs to me that, if we made a switch towards pipelining the function scan results directly out, then we might be loose efficiency where there are a significant number of scans and/or the function cost high. Is that why you were suggesting to as well stash them aside?

denty.

[1] https://www.postgresql.org/message-id/B2AFCAB5-FACD-44BF-963F-7DD2735FAB5D%40QQdd.eu