Adding pipelining support to set returning functions

From: Hannu Krosing <hannu(at)krosing(dot)net>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Adding pipelining support to set returning functions
Date: 2008-04-06 07:01:20
Message-ID: 1207465280.6904.20.camel@huvostro
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

A question to all pg hackers

Is anybody working on adding pipelining to set returning functions.

How much effort would it take ?

Where should I start digging ?

BACKGROUND:

AFAICS , currently set returning functions materialise their results
before returning, as seen by this simple test:

hannu=# select * from generate_series(1,10) limit 2;
generate_series
-----------------
1
2
(2 rows)

Time: 1.183 ms

hannu=# select * from generate_series(1,10000000) limit 2;
generate_series
-----------------
1
2
(2 rows)

Time: 3795.032 ms

being able to pipeline (generate results as needed) would enable several
interesting techniques, especially if combined with pl/proxy or any
other functions which stream external data.

Applications and design patterns like http://telegraph.cs.berkeley.edu/
or http://labs.google.com/papers/mapreduce.html would suddenly become
very easy to implement.

-----------------
Hannu

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Kenneth Marshall 2008-04-06 15:45:14 Re: machine-dependent hash_any vs the regression tests
Previous Message Marc G. Fournier 2008-04-06 02:27:25 Re: Archive email is not searchable