Linux I/O schedulers - CFQ & random seeks

Hi Guys,

I'm in the process of setting up some new hardware and am just doing some basic disk performance testing with bonnie++ to start with.

I'm seeing a massive difference on the random seeks test, with CFQ not performing very well as far as I can see. The thing is I didn't see this sort of massive divide when doing tests with our current hardware.

Current hardware: 2x4core E5420 @2.5Ghz/ 32GB RAM/ Adaptec 5805Z w' 512Mb/ Raid 10/ 8 15k 3.5 Disks
New hardware: 4x8core X7550 @2.0Ghz/ 128GB RAM/ H700 w' 1GB/ Raid 10/ 12 15.2k 2.5 Disks

Admittedly, my testing on our current hardware was on 2.6.26 and on the new hardware it's on 2.6.32 - I think I'm going to have to check the current hardware on the older kernel too.

I'm wondering (and this may be a can of worms) what peoples opinions are on these schedulers? I'm going to have to do some real world testing myself with postgresql too, but initially was thinking of switching from our current CFQ back to deadline.

Any opinions would be appreciated.

Regardless, here are some sample results from the new hardware:

CFQ:

Version 1.96 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
Way5ax 258376M 666 99 434709 96 225498 35 2840 69 952115 76 556.2 3
Latency 12344us 619ms 522ms 255ms 425ms 529ms
Version 1.96 ------Sequential Create------ --------Random Create--------
Way5ax -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 28808 41 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency 6170us 594us 633us 7619us 20us 36us
1.96,1.96,Way5ax,1,1299173113,258376M,,666,99,434709,96,225498,35,2840,69,952115,76,556.2,3,16,,,,,28808,41,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,12344us,619ms,522ms,255ms,425ms,529ms,6170us,594us,633us,7619us,20us,36us

deadline:

Version 1.96 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
Way5ax 258376M 696 99 449914 96 287010 47 2952 69 989527 78 2304 19
Latency 11939us 856ms 570ms 174ms 228ms 24744us
Version 1.96 ------Sequential Create------ --------Random Create--------
Way5ax -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 31338 45 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency 5605us 605us 627us 6590us 19us 38us
1.96,1.96,Way5ax,1,1299237441,258376M,,696,99,449914,96,287010,47,2952,69,989527,78,2304,19,16,,,,,31338,45,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,11939us,856ms,570ms,174ms,228ms,24744us,5605us,605us,627us,6590us,19us,38us

no-op:

Version 1.96 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
Way5ax 258376M 706 99 451578 95 303351 49 4104 96 1003688 78 2294 19
Latency 11538us 530ms 1460ms 12141us 350ms 22969us
Version 1.96 ------Sequential Create------ --------Random Create--------
Way5ax -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 31137 44 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency 5918us 597us 627us 5039us 17us 36us
1.96,1.96,Way5ax,1,1299245225,258376M,,706,99,451578,95,303351,49,4104,96,1003688,78,2294,19,16,,,,,31137,44,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,11538us,530ms,1460ms,12141us,350ms,22969us,5918us,597us,627us,5039us,17us,36us

--
Glyn