From: | Hannu Krosing <hannu(at)krosing(dot)net> |
---|---|
To: | pgsql-hackers(at)postgresql(dot)org |
Subject: | Adding pipelining support to set returning functions |
Date: | 2008-04-06 07:01:20 |
Message-ID: | 1207465280.6904.20.camel@huvostro |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
A question to all pg hackers
Is anybody working on adding pipelining to set returning functions.
How much effort would it take ?
Where should I start digging ?
BACKGROUND:
AFAICS , currently set returning functions materialise their results
before returning, as seen by this simple test:
hannu=# select * from generate_series(1,10) limit 2;
generate_series
-----------------
1
2
(2 rows)
Time: 1.183 ms
hannu=# select * from generate_series(1,10000000) limit 2;
generate_series
-----------------
1
2
(2 rows)
Time: 3795.032 ms
being able to pipeline (generate results as needed) would enable several
interesting techniques, especially if combined with pl/proxy or any
other functions which stream external data.
Applications and design patterns like http://telegraph.cs.berkeley.edu/
or http://labs.google.com/papers/mapreduce.html would suddenly become
very easy to implement.
-----------------
Hannu
From | Date | Subject | |
---|---|---|---|
Next Message | Kenneth Marshall | 2008-04-06 15:45:14 | Re: machine-dependent hash_any vs the regression tests |
Previous Message | Marc G. Fournier | 2008-04-06 02:27:25 | Re: Archive email is not searchable |