Re: Streaming LIBPQ? Sliding Window in LIBPQ?

On Wed, Nov 28, 2007 at 03:27:56PM +0000, Gregory Stark wrote:
> > Use non-blocking mode to get the data in chunks. You can't easy control
> > the number of rows you get each time though...
>
> This doesn't really help. It's nonblocking but you still can't get libpq to
> actually give you a result data structure until the entire results have
> accumulated.

It certainly used to work. You get a whole PGresult structure for each
few rows usually so it's not terribly efficient. I posted an example in
Perl a while back...

The trick was to set non-blocking mode and send an async query. Then
PQisBusy() would return false when any data had been received, not just
when all data had been received. At that point you could call
PQgetResult to get those rows. You would get a zero-length result when
you reached the end of data.

Admittedly, I havn't tested it on recent versions. The program I posted
a while back that tested if the locks blocked as documented drove two
connections simultaneously this way.

http://archives.postgresql.org/pgsql-hackers/2005-08/msg01073.php
http://archives.postgresql.org/pgsql-general/2006-07/msg00806.php

Have a nice day,
--
Martijn van Oosterhout <kleptog(at)svana(dot)org> http://svana.org/kleptog/
> Those who make peaceful revolution impossible will make violent revolution inevitable.
> -- John F Kennedy