From: | Daniele Varrazzo <daniele(dot)varrazzo(at)gmail(dot)com> |
---|---|
To: | AlanWEvans(at)gmail(dot)com |
Cc: | "psycopg(at)postgresql(dot)org" <psycopg(at)postgresql(dot)org> |
Subject: | Re: PQntuples for older version (2.2.1) of psycopg2 with ctypes |
Date: | 2016-02-20 13:14:29 |
Message-ID: | CA+mi_8aKJGZeebAZtddK4ueMSHYBdxsWpDYvhbjJC=YSgd=1Yg@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | psycopg |
On Fri, Feb 19, 2016 at 11:34 PM, Alan Evans <alanwevans(at)gmail(dot)com> wrote:
> I am using cursor.copy_expert and I would like to be able to use
> cursor.rowcount but most of the systems I am targeting have older psycopg2
> versions on them.
>
> According to the following URL rowcount was updated for use with copy_expert
> in 2.5.3...
> https://psycopg.lighthouseapp.com/projects/62710/tickets/180-return-number-of-rows-for-copy-operations
>
> Inspired by:
> https://github.com/sunlightlabs/psycopg2-ctypes/blob/master/psycopg2ct/_impl/cursor.py
>
> I wonder if it is possible to use ctypes to do this... Something like this
> maybe:
>
> import psycopg2
> import ctypes
> libpq = ctypes.CDLL('libpq.so.5')
>
> conn = psycopg2.connect(foo)
> cursor = conn.cursor()
> cursor.copy_expert(bar)
>
> rowcount = libpq.PQntuples(???)
>
> I can "read" the psycopg2 source but I am not fluent. I am having trouble
> working out what 'pgres' is. Is it the fileno() the backend_process_id()
> etc...
>
> So far all I have manged to do is segfault my python processes. Woot!
>
> Is 'pgres' exposed by the Cursor or Connection classes somewhere?
Funny hack :)
The pgres is on the cursor:
<https://github.com/psycopg/psycopg2/blob/2_2_1/psycopg/cursor.h#L60>.
Thankfully (at least in 2.2.1 but I guess it never changed) it is
located right after the cur.description pointer, so we can use it to
find the offset pgres offset in the structure:
import psycopg2
cnn = psycopg2.connect('')
cur = cnn.cursor()
cur.execute("select 1")
import ctypes, struct, sys
curdump = ctypes.string_at(id(cur), sys.getsizeof(cur))
descptr = struct.pack('P', id(cur.description))
resoff = curdump.find(descptr) + len(descptr)
The offset is fixed for a given build so it won't change at runtime.
You can inspect the cursor after the COPY operation to find the result
structure and pass it to the libpq:
libpq = ctypes.cdll.LoadLibrary("libpq.so")
cur.execute("select generate_series(1,42)") # cur.copy_expert here
curdump = ctypes.string_at(id(cur), sys.getsizeof(cur))
resptr = struct.unpack('P', curdump[resoff:resoff+len(descptr)])[0]
print libpq.PQntuples(resptr)
# 42
Hope this helps :)
-- Daniele
From | Date | Subject | |
---|---|---|---|
Next Message | Daniele Varrazzo | 2016-02-20 13:25:56 | Re: How to use "WHERE NOT EXISTS in INSERT STATEMENT" |
Previous Message | Jonathan Rogers | 2016-02-20 12:34:27 | Re: How to use "WHERE NOT EXISTS in INSERT STATEMENT" |