Re: libpq single-row mode performance

Hello, James.

You wrote:

JD> Hi,
JD>
JD> I’m writing an app on top of libpq. To avoid running out of
JD> memory I’m using the single-row mode, but am finding the overhead in using this quite significant.
JD>
JD> What I see is that ~60% of my application’s run time when
JD> retrieving data is spent in calls to either PQgetResult() and
JD> PQclear(). I’ve added a switch in my app to turn off single-row
JD> mode and the performance roughly doubles.
JD>
JD> Would it be possible to optimize the single-row mode? For
JD> example, add an API call PQnextRow(PGConn*), which is only usable
JD> in single-row mode and will just update the existing result with
JD> the contents of the next row? This would let us avoid to overhead of:
JD> 1. Malloc’ing a PGResult and initializing its defaults.
JD> 2. Copying the column metadata to the new results.
JD> 3. Possibly we can avoid malloc’ing cell data if the next
JD> row has cells the same size or smaller than a previous row, though
JD> I’m not sure of the internals here.
JD> 4. Release memory with PQclear().
JD>
JD> James Duong | Senior Computer Scientist | Simba Technologies Inc.
JD> Tel +1.604.633.0008 ext. 120 | Fax +1.604.633.0004 | jamesd(at)simba(dot)com
JD>
JD> 938 West 8th Avenue | Vancouver, BC | Canada | V5Z 1E5
JD> The Data Access and Analytics Experts | www.simba.com
JD>
JD>
JD>
JD> This email message is for the sole use of the intended
JD> recipient(s) and may contain confidential and privileged
JD> information. Any unauthorized review, use, disclosure, or
JD> distribution is prohibited. If you are not the intended
JD> recipient, please contact the sender by reply email and destroy
JD> all copies of the original message. Thank you.
JD>

Good idea. +1 from me. Should we send this message to pg-hackers as
well?

--
With best wishes,
Pavel mailto:pavel(at)gf(dot)microolap(dot)com