Re: 9.5: Better memory accounting, towards memory-bounded HashAgg

From: Jeff Davis <pgsql(at)j-davis(dot)com>
To: Tomas Vondra <tv(at)fuzzy(dot)cz>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: 9.5: Better memory accounting, towards memory-bounded HashAgg
Date: 2014-12-28 21:29:43
Message-ID: 1419802183.24895.81.camel@jeff-desktop
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Sun, 2014-12-28 at 12:37 -0800, Jeff Davis wrote:
> I feel like I made a mistake -- can someone please do a
> sanity check on my numbers?

I forgot to randomize the inputs, which doesn't matter much for hashagg
but does matter for sort. New data script attached. The results are even
*better* for disk-based hashagg than the previous numbers suggest. Here
are some new numbers:

work_mem='1MB':
sort+group (s) hashagg (s)
singleton q1 21 10
singleton q2 12 8
even q1 20 7
even q2 13 5
skew q1 22 6
skew q2 16 4

work_mem='4MB':
sort+group (s) hashagg (s)
singleton q1 17 10
singleton q2 11 6
even q1 16 7
even q2 11 5
skew q1 19 6
skew q2 13 4

work_mem='16MB':
sort+group (s) hashagg (s)
singleton q1 16 11
singleton q2 11 7
even q1 15 8
even q2 12 6
skew q1 15 6
skew q2 12 4

work_mem='64MB':
sort+group (s) hashagg (s)
singleton q1 18 12
singleton q2 13 8
even q1 17 10
even q2 13 6
skew q1 17 6
skew q2 14 4

work_mem='256MB':
sort+group (s) hashagg (s)
singleton q1 18 12
singleton q2 14 7
even q1 16 9
even q2 14 5
skew q1 18 6
skew q2 13 4

work_mem='512MB':
sort+group (s) hashagg (s)
singleton q1 18 12
singleton q2 14 7
even q1 17 9
even q2 14 5
skew q1 17 6
skew q2 13 4

work_mem='2GB':
sort+group (s) hashagg (s)
singleton q1 11 11
singleton q2 7 6
even q1 10 9
even q2 7 5
skew q1 7 6
skew q2 4 4

Regards,
Jeff Davis

Attachment Content-Type Size
hashagg_test_data.sql application/sql 1.3 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Noah Misch 2014-12-28 21:58:31 Re: orangutan seizes up during isolation-check
Previous Message Jeff Davis 2014-12-28 21:14:50 Re: 9.5: Memory-bounded HashAgg