Re: Recursive calls to functions that return sets

On Wed, Mar 22, 2006 at 03:31:59PM +0100, Thomas Hallgren wrote:
> Imagine the following scenario:
>
> Function 'A' returns SETOF 'x'. It will issue a query using SPI that
> calls function 'B'. This function returns SETOF 'y'.
> Each tuple of 'x' is formed from some data in 'y'.
> There will be millions of tuples so building a set of 'y' in memory is
> not an option.

I think you're running into a small limitation of set functions here.
If you look at nodeFunctionScan.c that handles this, you can see that
the code is written in such a way as to collect all the tuples first
before returning anything. Not sure why it does that, probably to
handle mark/restore, though that isn't stated anywhere in the code.

> What would the recommended use of MemoryContexts in an SRF function be
> in order to make this work? The SPI_connect must be issued during the
> SRF_IS_FIRST_CALL() phase. The Portal that it creates must remain alive
> until it's time for the SRF_RETURN_DONE(). What would the recommended
> approach be to accomplish this efficiently (and without introducing a
> major memory leak)?

Well, I think this is done the normal way. The function returning
values allocates them in it's own context and does a RETURN NEXT. Once
it has returned them it can free it, or reset the context if it
prefers. The caller is always responsible for copying (since it isn't
often needed).

Have you read the executor/README ?

Have a nice day,
--
Martijn van Oosterhout <kleptog(at)svana(dot)org> http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.