plpython SPI cursors

From: Jan Urbański <wulczer(at)wulczer(dot)org>
To: Postgres - Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: plpython SPI cursors
Date: 2011-10-15 23:28:56
Message-ID: 4E9A1738.6090108@wulczer.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

attached is a patch implementing the usage of SPI cursors in PL/Python.
Currently when trying to process a large table in PL/Python you have
slurp it all into memory (that's what plpy.execute does).

This patch allows reading the result set in smaller chunks, using a SPI
cursor behind the scenes.

Example usage:

cursor = plpy.cursor("select a, b from hugetable")
for row in cursor:
plpy.info("a is %s and b is %s" % (row['a'], row['b']))

The patch itself is simple, but there's a lot of boilerplate dedicated
to opening a subtransaction and handling prepared plans. I'd like to do
some refactoring of they way PL/Python uses SPI to reduce the amount of
boilerplate needed, but that'll come as a separate patch (just before
the patch to split plpython.c in smaller chunks).

This feature has been sponsored by Nomao.

Cheers,
Jan

PS: I already added it to the November CF.

J

Attachment Content-Type Size
0001-Add-cursor-support-to-plpythonu.patch text/x-diff 0 bytes

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Chris Redekop 2011-10-16 01:33:33 Re: Hot Backup with rsync fails at pg_clog if under load
Previous Message Jeff Janes 2011-10-15 22:58:49 Re: COUNT(*) and index-only scans