Re: Use generation context to speed up tuplesorts

On 8/8/21 12:11 PM, David Rowley wrote:
> On Sat, 7 Aug 2021 at 12:10, Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com> wrote:
>>> All of the tests show that the patches to improve the allocation
>>> efficiency of generation.c don't help to improve the results of the
>>> test cases. I wondered if it's maybe worth trying to see what happens
>>> if instead of doubling the allocations each time, quadruple them
>>> instead. I didn't try this.
>>>
>>
>> I doubt quadrupling the allocations won't help very much, but I suspect
>> the problem might be in the 0004 patch - at least that's what shows
>> regression in my results. Could you try with just 0001-0003 applied?
>
> I tried the quadrupling of the buffer instead of doubling it each time
> and got the attached. Column E, or green in the graphs show the
> results of that. It's now much closer to the original patch which just
> made the block size 8MB.
>

Interesting, I wouldn't have expected that to make such difference.

I'm not sure quadrupling the size is a good idea, though, because it
increases the amount of memory we might be wasting. With the doubling,
the amount of wasted /unused memory is limited to ~50%, because the next
block is (roughly) equal to sum of already allocated blocks, so
allocating just 1B on it leaves us with 50%. But quadrupling the size
means we'll end up with ~75% free space. Of course, this is capped by
the maximum block size etc. but still ...

regards

--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company