From: | Craig Ringer <craig(at)2ndquadrant(dot)com> |
---|---|
To: | Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com> |
Cc: | Pg Hackers <pgsql-hackers(at)postgresql(dot)org>, Joe Conway <mail(at)joeconway(dot)com> |
Subject: | Re: Faster methods for getting SPI results (460% improvement) |
Date: | 2017-01-24 04:36:35 |
Message-ID: | CAMsr+YGC9qpxeLhLHXA6a=ztz-DZ2AE2p2wkd9svj7wC1eHngA@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On 24 January 2017 at 11:23, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com> wrote:
> I finally got all the kinks worked out and did some testing with python 3.
> Performance for my test [1] improved ~460% when returning a dict of lists
> (as opposed to the current list of dicts). Based on previous testing, I
> expect that using this method to return a list of dicts will be about 8%
> slower. The inconsistency in results on 2.7 has to do with how python 2
> handles ints.
Impressive results.
> I think the last step here is to figure out how to support switching between
> the current behavior and the "columnar" behavior of a dict of lists.
That sounds like it'd be much better approached as a separate, later patch.
If I understand you correctly, you propose to return the resultset
a b
1 10
2 20
which is currently returned as
[ {"a":1, "b":10}, {"a":2, "b":20} ]
instead as
{ "a": [1, 2], "b": [10, 20] }
?
If so I see that as a lot more of a niche thing. I can see why it'd be
useful and would help performance, but it seems much more disruptive.
It requires users to discover it exists, actively adopt a different
style of ingesting data, etc. For a 10%-ish gain in a PL.
I strongly suggest making this design effort a separate thread, and
focusing on the SPI improvements that give "free" no-user-action
performance boosts here.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
From | Date | Subject | |
---|---|---|---|
Next Message | Ashutosh Sharma | 2017-01-24 04:39:58 | Re: Add pgstathashindex() to get hash index table statistics. |
Previous Message | Nico Williams | 2017-01-24 04:10:19 | Re: Contrib: pqasyncnotifier.c -- a shell command client for LISTEN |