Re: things I learned from working on memory allocation

On Thu, Jul 10, 2014 at 1:05 AM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> On Tue, Jul 8, 2014 at 1:27 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>> 6. In general, I'm worried that it's going to be hard to keep the
>> overhead of parallel sort from leaking into the non-parallel case.
>> With the no-allocator approach, every place that uses
>> GetMemoryChunkSpace() or repalloc() or pfree() will have to handle the
>> DSM and non-DSM cases differently, which isn't great for either
>> performance or maintainability. And even with an allocator, the
>> SortTuple array will need to use relative pointers in a DSM; that
>> might burden the non-DSM case.
>
> I think you are right in saying that there can be a performance impact
> for non-parallel case due to pfree() (or other similar calls) and/or due
> to usage of relative pointers, but can it have measurable impact during
> actual sort operation?

Benchmarking seems to indicate that it indeed can.

> If there is an noticeable impact, then do you think having separate
> file/infrastructure for parallel sort can help, basically non-parallel
> and parallel sort will have some common and some specific parts.
> The above layer will call the parallel/non-parallel functions based on
> operation need.

The tuplesort.c already handles so many cases already that adding yet
another axis of variability is intimidating. But it may work out that
there's no better option.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company