Re: [HACKERS] sort on huge table

--- Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Tatsuo Ishii <t-ishii(at)sra(dot)co(dot)jp> writes:
> > RedHat Linux 6.0 (kernel 2.2.5-smp)
> > Pentium III 500MHz x 2
> > RAM: 512MB
> > Disk: Ultra Wide SCSI 9GB x 4 + Hardware RAID (RAID 5).
>
> OK, no problem with inadequate hardware anyway ;-). Bruce's concern
> about simplistic read-ahead algorithm in Linux may apply though.
>
> > Also, I could provide testing scripts to reproduce my tests.
>
> Please. That would be very handy so that we can make sure we are all
> comparing the same thing. I assume the scripts can be tweaked to vary
> the amount of disk space used? I can't scare up more than a couple
> hundred meg at the moment. (The natural state of a disk drive is
> "full" ...)
>
> > I think it depends on the disk space available. Ideally it should be
> > able to choice the sort algorithm.
>
> I was hoping to avoid that, because of the extra difficulty of testing
> and maintenance. But it may be the only answer.
>
> regards, tom lane

I know this is a VERY long shot, but... what were the READ/WRITE ratios
between the old version and the new version? Perhaps the computation
of the checksum (sic) blocks under RAID5 caused the unexpected behavior.
With RAID 5 increasing read performance but decreasing writes, one might
expect a new algorithm which say, halves reads, but increases writes
slightly to not realize the same benefits as under a normal disk system or
a RAID 1 (or, better yet, a RAID 0+1) array.

Like I said...a VERY long shot theory.

Mike Mascari
(mascarim(at)yahoo(dot)com)

=====

__________________________________________________
Do You Yahoo!?
Bid and sell for free at http://auctions.yahoo.com