From: | Pavel Golub <pavel(at)microolap(dot)com> |
---|---|
To: | James Duong <jamesd(at)simba(dot)com> |
Cc: | "pgsql-interfaces(at)postgresql(dot)org" <pgsql-interfaces(at)postgresql(dot)org> |
Subject: | Re: libpq single-row mode performance |
Date: | 2014-07-03 12:27:49 |
Message-ID: | 13210629610.20140703152749@gf.microolap.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-interfaces |
Hello, James.
You wrote:
JD> Hi,
JD>
JD> I’m writing an app on top of libpq. To avoid running out of
JD> memory I’m using the single-row mode, but am finding the overhead in using this quite significant.
JD>
JD> What I see is that ~60% of my application’s run time when
JD> retrieving data is spent in calls to either PQgetResult() and
JD> PQclear(). I’ve added a switch in my app to turn off single-row
JD> mode and the performance roughly doubles.
JD>
JD> Would it be possible to optimize the single-row mode? For
JD> example, add an API call PQnextRow(PGConn*), which is only usable
JD> in single-row mode and will just update the existing result with
JD> the contents of the next row? This would let us avoid to overhead of:
JD> 1. Malloc’ing a PGResult and initializing its defaults.
JD> 2. Copying the column metadata to the new results.
JD> 3. Possibly we can avoid malloc’ing cell data if the next
JD> row has cells the same size or smaller than a previous row, though
JD> I’m not sure of the internals here.
JD> 4. Release memory with PQclear().
JD>
JD> James Duong | Senior Computer Scientist | Simba Technologies Inc.
JD> Tel +1.604.633.0008 ext. 120 | Fax +1.604.633.0004 | jamesd(at)simba(dot)com
JD>
JD> 938 West 8th Avenue | Vancouver, BC | Canada | V5Z 1E5
JD> The Data Access and Analytics Experts | www.simba.com
JD>
JD>
JD>
JD> This email message is for the sole use of the intended
JD> recipient(s) and may contain confidential and privileged
JD> information. Any unauthorized review, use, disclosure, or
JD> distribution is prohibited. If you are not the intended
JD> recipient, please contact the sender by reply email and destroy
JD> all copies of the original message. Thank you.
JD>
Good idea. +1 from me. Should we send this message to pg-hackers as
well?
--
With best wishes,
Pavel mailto:pavel(at)gf(dot)microolap(dot)com
From | Date | Subject | |
---|---|---|---|
Next Message | madhu_d | 2014-07-06 23:29:18 | Insert/update a column if it doesnot exists in a database using C program |
Previous Message | James Duong | 2014-07-03 01:05:32 | libpq single-row mode performance |