| From: | Hannu Krosing <hannu(at)krosing(dot)net> |
|---|---|
| To: | pgsql-hackers(at)postgresql(dot)org |
| Subject: | Adding pipelining support to set returning functions |
| Date: | 2008-04-06 07:01:20 |
| Message-ID: | 1207465280.6904.20.camel@huvostro |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
A question to all pg hackers
Is anybody working on adding pipelining to set returning functions.
How much effort would it take ?
Where should I start digging ?
BACKGROUND:
AFAICS , currently set returning functions materialise their results
before returning, as seen by this simple test:
hannu=# select * from generate_series(1,10) limit 2;
generate_series
-----------------
1
2
(2 rows)
Time: 1.183 ms
hannu=# select * from generate_series(1,10000000) limit 2;
generate_series
-----------------
1
2
(2 rows)
Time: 3795.032 ms
being able to pipeline (generate results as needed) would enable several
interesting techniques, especially if combined with pl/proxy or any
other functions which stream external data.
Applications and design patterns like http://telegraph.cs.berkeley.edu/
or http://labs.google.com/papers/mapreduce.html would suddenly become
very easy to implement.
-----------------
Hannu
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Kenneth Marshall | 2008-04-06 15:45:14 | Re: machine-dependent hash_any vs the regression tests |
| Previous Message | Marc G. Fournier | 2008-04-06 02:27:25 | Re: Archive email is not searchable |