Re: Using quicksort for every external sort run

On Wed, Mar 16, 2016 at 9:42 PM, Peter Geoghegan <pg(at)heroku(dot)com> wrote:
>> - I haven't yet figured out why we use batching only for the final
>> on-the-fly merge pass, instead of doing it for all merges. I expect
>> you have a reason. I just don't know what it is.
>
> The most obvious reason, and possibly the only reason, is that I have
> license to lock down memory accounting in the final on-the-fly merge
> phase. Almost equi-sized runs are the norm, and code like this is no
> longer obligated to work:
>
> FREEMEM(state, GetMemoryChunkSpace(stup->tuple));
>
> That's why I explicitly give up on "conventional accounting". USEMEM()
> and FREEMEM() calls become unnecessary for this case that is well
> locked down. Oh, and I know that I won't use most tapes, so I can give
> myself a FREEMEM() refund before doing the new grow_memtuples() thing.
>
> I want to make batch memory usable for runs, too. I haven't done that
> either for similar reasons. FWIW, I see no great reason to worry about
> non-final merges.

Fair enough. My concern was mostly whether the code would become
simpler if we always did this when merging, instead of only on the
final merge. But the final merge seems to be special in quite a few
respects, so maybe not.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company