Re: Scalability in postgres

Thanks Scott

2009/5/28 Scott Marlowe <scott(dot)marlowe(at)gmail(dot)com>

> On Thu, May 28, 2009 at 12:50 PM, Fabrix <fabrixio1(at)gmail(dot)com> wrote:
> >
> > HI.
> >
> > Someone had some experience of bad performance with postgres in some
> server
> > with many processors?
>
> Seems to depend on the processors and chipset a fair bit.
>
> > I have a server with 4 CPUS dual core and gives me a very good
> performance
> > but I have experienced problems with another server that has 8 CPUS quad
> > core (32 cores). The second one only gives me about 1.5 of performance of
> > the first one.
>
> What model CPUs and chipset on the mobo I wonder?

I have 2 Servers of this type in which I tested, an HP and IBM. HP gives me
better performance, IBM already discarded

Server HP:
cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 16
model : 2
model name : Quad-Core AMD Opteron(tm) Processor 8360 SE
stepping : 3
cpu MHz : 2500.091
cache size : 512 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 4
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush mmx fxsr sse sse2 ht syscall mmxext fxsr_opt pdpe1gb
rdtscp lm 3dnowext 3dnow constant_tsc pni cx16 popcnt lahf_lm cmp_legacy svm
extapic cr8_legacy altmovcr8 abm sse4a misalignsse 3dnowprefetch osvw
bogomips : 5004.94
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate [8]

Server IBM:
cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Xeon(R) CPU X7350 @ 2.93GHz
stepping : 11
cpu MHz : 2931.951
cache size : 4096 KB
physical id : 3
siblings : 4
core id : 0
cpu cores : 4
apicid : 12
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm
constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm
bogomips : 5867.00
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:

>
> > Monitoring (nmon, htop, vmstat) see that everything is fine (memory, HD,
> > eth, etc) except that processors regularly climb to 100%.
> >
> > I can see that the processes are waiting for CPU time:
> >
> > vmstat 1
> >
> > procs -----------memory---------- ---swap-- -----io---- --system--
> > -----cpu------
> > r b swpd free buff cache si so bi bo in cs us sy
> id
> > wa st
> > 0 0 0 47123020 117320 17141932 0 0 8 416 1548 2189 1
> 1
> > 99 0 0
> > 0 0 0 47121904 117328 17141940 0 0 8 148 1428 2107 1
> 1
> > 98 0 0
> > 0 0 0 47123144 117336 17141956 0 0 8 172 1391 1930 1
> 0
> > 99 0 0
> > 0 0 0 47124756 117352 17141940 0 0 8 276 1327 2171 1
> 1
> > 98 0 0
> > 0 0 0 47118556 117360 17141956 0 0 0 100 1452 2254 1
> 1
> > 98 0 0
> > 2 0 0 47120364 117380 17141952 0 0 8 428 1526 2477 1
> 0
> > 99 0 0
> > 1 0 0 47119372 117388 17141972 0 0 0 452 1581 2662 1
> 1
> > 98 0 0
> > 0 0 0 47117948 117396 17141988 0 0 16 468 1705 3243 1
> 1
> > 97 0 0
> > 0 0 0 47116708 117404 17142020 0 0 0 268 1610 2115 1
> 1
> > 99 0 0
> > 0 0 0 47119688 117420 17142044 0 0 0 200 1545 1810 1
> 1
> > 98 0 0
> > 318 0 0 47116464 117440 17142052 0 0 0 532 1416 2396
> 1 0
> > 99 0 0
> > 500 0 0 47115224 117440 17142052 0 0 0 0 1118 322144
> 91
> > 5 4 0 0
> > 440 0 0 47114728 117440 17142044 0 0 0 0 1052 333137
> 90
> > 5 5 0 0
> > 339 0 0 47114484 117440 17142048 0 0 0 0 1061 337528
> 85
> > 4 11 0 0
> > 179 0 0 47114112 117440 17142048 0 0 0 0 1066 312873
> 71
> > 4 25 0 0
> > 5 1 0 47122180 117468 17142028 0 0 192 3128 1958 136804
> 23
> > 2 75 1 0
> > 3 0 0 47114264 117476 17142968 0 0 608 5828 2688 4684 7
> 2
> > 89 2 0
> > 0 1 0 47109940 117484 17142876 0 0 512 5084 2248 3727 3
> 1
> > 94 2 0
> > 0 1 0 47119692 117500 17143816 0 0 520 4976 2231 2941 2
> 1
> > 95 2 0
> >
> >
> > Have postgres problems of lock or degradation of performance with many
> > CPU's?
> > Any comments?
>
> Looks like a context switch storm, which was pretty common on older
> Xeon CPUs. I imagine with enough pg processes running on enough CPUs
> it could still be a problem.

These CPUs are very new.