Re: Dynamic Shared Memory stuff

On 20/11/13 19:58, Robert Haas wrote:
> Parallel sort, and then parallel other stuff. Eventually general
> parallel query.
>
> I have recently updated https://wiki.postgresql.org/wiki/Parallel_Sort
> and you may find that interesting/helpful as a statement of intent.

I've been playing with an internal merge sort which may be of interest
in this area of work. While I've not worked on parallelizing the merge
stages it does not feel like an impossible task. More to the immediate
point, the input stage and readout stage can do useful work
overlapped with the data source and sink (respectively).

The current implementation uses the same amount of memory as the
quicksort one, and does approximately the same number of compares
(on random input). The overheads are higher than the quicksort
implementation (up to 50% higher cpu time, on a single key of
random integers).

Its performance shines on partially- or reverse-sorted input.

Transition to (the existing) external sort is implemented, as is
the limited random-access facility, and bounded output.

It supports unique-output. The planner interface is extended to
return an indication of dedup guarantee, and the Unique node uses
this to elide itself at planning time. Dedup operations also
cover external sorts.
--
Cheers,
Jeremy