Re: Two identical systems, radically different performance

On Mon, Oct 8, 2012 at 3:33 PM, Evgeny Shishkin <itparanoia(at)gmail(dot)com>wrote:

>
> On Oct 9, 2012, at 1:45 AM, Craig James <cjames(at)emolecules(dot)com> wrote:
>
> One dramatic difference I noted via vmstat. On the old server, the I/O
> load during the bonnie++ run was steady, like this:
>
> procs -----------memory---------- ---swap-- -----io---- -system--
> ----cpu----
> r b swpd free buff cache si so bi bo in cs us sy id
> wa
> r b swpd free buff cache si so bi bo in cs us sy id
> wa
> 0 2 71800 2117612 17940 9375660 0 0 82948 81944 1992 1341 1 3
> 86 10
> 0 2 71800 2113328 17948 9383896 0 0 76288 75806 1751 1167 0 2
> 86 11
> 0 1 71800 2111004 17948 9386540 92 0 93324 94232 2230 1510 0 4
> 86 10
> 0 1 71800 2106796 17948 9387436 114 0 67698 67588 1572 1088 0 2
> 87 11
> 0 1 71800 2106724 17956 9387968 50 0 81970 85710 1918 1287 0 3
> 86 10
> 1 1 71800 2103304 17956 9390700 0 0 92096 92160 1970 1194 0 4
> 86 10
> 0 2 71800 2103196 17976 9389204 0 0 70722 69680 1655 1116 1 3
> 86 10
> 1 1 71800 2099064 17980 9390824 0 0 57346 57348 1357 949 0 2
> 87 11
> 0 1 71800 2095596 17980 9392720 0 0 57344 57348 1379 987 0 2
> 86 12
>
> But the new server varied wildly during bonnie++:
>
> procs -----------memory---------- ---swap-- -----io---- -system--
> ----cpu----
> r b swpd free buff cache si so bi bo in cs us sy id
> wa
> 0 1 0 4518352 12004 7167000 0 0 118894 120838 2613 1539 0
> 2 93 5
> 0 1 0 4517252 12004 7167824 0 0 52116 53248 1179 793 0
> 1 94 5
> 0 1 0 4515864 12004 7169088 0 0 46764 49152 1104 733 0
> 1 91 7
> 0 1 0 4515180 12012 7169764 0 0 32924 30724 750 542 0
> 1 93 6
> 0 1 0 4514328 12016 7170780 0 0 42188 45056 1019 664 0
> 1 90 9
> 0 1 0 4513072 12016 7171856 0 0 67528 65540 1487 993 0
> 1 96 4
> 0 1 0 4510852 12016 7173160 0 0 56876 57344 1358 942 0
> 1 94 5
> 0 1 0 4500280 12044 7179924 0 0 91564 94220 2505 2504 1
> 2 91 6
> 0 1 0 4495564 12052 7183492 0 0 102660 104452 2289 1473 0
> 2 92 6
> 0 1 0 4492092 12052 7187720 0 0 98498 96274 2140 1385 0
> 2 93 5
> 0 1 0 4488608 12060 7190772 0 0 97628 100358 2176 1398 0
> 1 94 4
> 1 0 0 4485880 12052 7192600 0 0 112406 114686 2461 1509 0
> 3 90 7
> 1 0 0 4483424 12052 7195612 0 0 64678 65536 1449 948 0
> 1 91 8
> 0 1 0 4480252 12052 7199404 0 0 99608 100356 2217 1452 0
> 1 96 3
>
>
> Also note the difference in free/cache distribution. Unless you took these
> numbers in completely different stages of bonnie++.
>
>
The old server is in production and is running Apache/Postgres requests.

Craig