Performance Enhancement/Fix for Array Utility Functions

From: Mike Lewis <mikelikespie(at)gmail(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Performance Enhancement/Fix for Array Utility Functions
Date: 2010-03-31 07:39:57
Message-ID: z2x948ce0b71003310039gb3e62a1di32d81d96c437b7aa@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I noticed while doing work with very large arrays that several
functions such as array_length detoast the entire array instead of
only what is required.

I found the solution to be just unpacking the header portion of the
array and ignoring the rest. Since the header (including the
dimensions) is variable length, I just unpack the size of what the
header would be if it had MAXDIM dimensions. (Patch is attached)

I made a test case to demonstrate performance gains (watch out, it
creates a big table):

create temporary table foo as
select array_agg(i) as a
from (
select generate_series(1,10000000) as i) as bar;
\timing
select array_length(a, 1) from foo; -- Run a few times. First time will be cold

Results (after warming up)

Before patch:
Time: 6.251 ms
Time: 6.078 ms
Time: 5.983 ms

After patch:
Time: 0.401 ms
Time: 0.397 ms
Time: 0.441 ms
...

--
Michael Lewis
lolrus.org
mikelikespie(at)gmail(dot)com

Attachment Content-Type Size
detoast-headers-for-array-functions-001.patch application/octet-stream 5.6 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Dave Page 2010-03-31 07:57:48 Re: Alpha release this week?
Previous Message Thom Brown 2010-03-31 07:06:19 Re: [SPAM]Re: Questions about 9.0 release note