looping over the rows in a table

From: Rajarshi Guha <rguha(at)indiana(dot)edu>
To: pgsql-general <pgsql-general(at)postgresql(dot)org>
Subject: looping over the rows in a table
Date: 2007-11-10 00:12:07
Message-ID: 156BA837-4ABE-4612-B16C-CDFB0C9E69A6@indiana.edu
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi, this is slightly offtopic, but is based on Postgres:

I have a table with 10M rows and I have a Python script using psycopg
that needs to look at each row of the table. My current strategy is
to do in the Python script

cursor.execute("select acol from atable")
while True:
ret = cursor.fetchone()
if not ret: break

However if I understand correctly Postgres will basically try and
return *all* the rows of the table as the result set, thus taking a
long time and probably running out of memory.

Is there a way I can modify the SQL or do something on the Postgres
side, so that I can loop over all the rows in the table?

Thanks,

-------------------------------------------------------------------
Rajarshi Guha <rguha(at)indiana(dot)edu>
GPG Fingerprint: 0CCA 8EE2 2EEB 25E2 AB04 06F7 1BB9 E634 9B87 56EE
-------------------------------------------------------------------
A bug in the hand is better than one as yet undetected.

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Scott Marlowe 2007-11-10 01:14:59 Re: [HACKERS] Is "query" a reserved word in 8.3 plpgsql?
Previous Message Tom Lane 2007-11-10 00:07:17 Re: [HACKERS] Is "query" a reserved word in 8.3 plpgsql?