Re: Vacuum, Freeze and Analyze: the big picture

On Jun 3, 2013, at 6:45 PM, Craig Ringer <craig(at)2ndquadrant(dot)com> wrote:
> On 06/04/2013 05:27 AM, Peter Geoghegan wrote:
>> On Mon, Jun 3, 2013 at 5:03 AM, Craig Ringer <craig(at)2ndquadrant(dot)com> wrote:
>>> I've seen cases on Stack Overflow and elsewhere in which disk merge
>>> sorts perform vastly better than in-memory quicksort, so the user
>>> benefited from greatly *lowering* work_mem.
>> I've heard of that happening on Oracle, when the external sort is
>> capable of taking advantage of I/O parallelism, but I have a pretty
>> hard time believing that it could happen with Postgres under any
>> circumstances.
> IIRC it's usually occurred with very expensive comparison operations.
>
> I'll see if I can find one of the SO cases.

FWIW, I've definitely seen this behavior in the past, on really old versions (certainly pre-9, possibly pre-8).

IIRC there's some kind of compression or something used with on-disk sorts. If that's correct then I think what's happening is that the "on-disk" sort that fits into cache is actually using less memory than quicksort. Or perhaps it was just a matter of memory locality within each tape. It's been too long since I looked at it. :(