Re: Compression and on-disk sorting

On Fri, May 26, 2006 at 09:21:44PM +0100, Simon Riggs wrote:
> On Fri, 2006-05-26 at 14:47 -0500, Jim C. Nasby wrote:
>
> > But the meat is:
> > -- work_mem --
> > Scale 2000 20000
> > not compressed 150 805.7 797.7
> > not compressed 3000 17820 17436
> > compressed 150 371.4 400.1
> > compressed 3000 8152 8537
> > compressed, no headers 3000 7325 7876
>
> Since Tom has committed the header-removing patch, we need to test
>
> not compressed, no headers v compressed, no headers

-- work_mem --
Scale 2000 20000
not compressed 150 805.7 797.7
not compressed 3000 17820 17436
not compressed, no hdr 3000 14470 14507
compressed 150 371.4 400.1
compressed 3000 8152 8537
compressed, no headers 3000 7325 7876

> There is a noticeable rise in sort time with increasing work_mem, but
> that needs to be offset from the benefit that in-general comes from
> using a large Heap for the sort. With the data you're using that always
> looks like a loss, but that isn't true with all input data orderings.

I thought that a change had been made to the on-disk sort specifically to
eliminate the problem of more work_mem making the sort take longer. I also
thought that there was something about that fix that was tunable.
--
Jim C. Nasby, Sr. Engineering Consultant jnasby(at)pervasive(dot)com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461