Re: zLinux Load Testing Experience

On 01/05/13 19:21, Merlin Moncure wrote:
> On Wed, May 1, 2013 at 11:34 AM, Andrew Hastie <andrew(at)ahastie(dot)net> wrote:
>> On 01/05/13 15:34, Merlin Moncure wrote:
>>
>> On Wed, May 1, 2013 at 8:01 AM, Andrew Hastie <andrew(at)ahastie(dot)net> wrote:
>>
>> On 30/04/13 20:46, Merlin Moncure wrote:
>>
>> On Tue, Apr 30, 2013 at 12:26 PM, Jeff Janes <jeff(dot)janes(at)gmail(dot)com> wrote:
>>
>> On Tue, Apr 30, 2013 at 8:28 AM, Andrew Hastie <andrew(at)ahastie(dot)net>
>> wrote:
>>
>> I'm currently working on a project porting an application from RedHat
>> Linux on Intel onto IBM zLinux. Our application requires PostgreSQL at
>> version 9.n, so the PostgreSQL binaries have been built using the
>> standard
>> build tools from source. Everything appears run correctly. However as
>> part
>> of performance testing, our IBM and Linux SysProgs have been "poking
>> around"
>> using strace and have reported the following (which they think is an
>> error
>> condition) when hooking up to the postmaster processes:-
>>
>> read(3, 0x3ffff875ee0, 16) = -1 EAGAIN (Resource temporarily
>> unavailable)
>> poll([{fd=3, events=POLLIN}, {fd=6, events=POLLIN}], 2, 200) = 0
>> (Timeout)
>> read(3, 0x3ffff875ee0, 16) = -1 EAGAIN (Resource temporarily
>> unavailable)
>> poll([{fd=3, events=POLLIN}, {fd=6, events=POLLIN}], 2, 10000) = 0
>> (Timeout)
>> ... repeated many times
>>
>> That does not look like the postmaster process. It looks like probably
>> the
>> background writer process.
>>
>> It is normal, and doesn't explain high CPU utilization.
>>
>> yeah: we're probably a couple of steps in front of deep system
>> profiling. Helpful things to provide to help diagnose would be:
>>
>> *) 'explain analyze' of the queries that are eating cpu
>> *) more details about the hardware -- how many cpu, etc.
>> *) better definition of 'perceived high CPU utilisation'
>> *) some correlating performance tests, expecially cpu bound pgbench
>> tests (pgbench -S)
>>
>> merlin
>>
>>
>> I'm not sure how much experience the community has on tuning PostgreSQL
>> running on RedHat which in turn is hosted on an IBM mainframe under VM
>> (using zLinux). So I'm happy to start posting further details and benchmark
>> results and see where we go. Should I be moving this thread over into the
>> pg-performance list, or is pg-general the right place?
>>
>> certainly performance. and yes, zLinux is less well traveled. Did
>> you compile postgres from source? Did you confirm that there is a
>> native spinlocks implementation and it is being used?
>>
>> merlin
>>
>> Did you compile postgres from source? - Yes (I need PG v9.n as v8.n shipped
>> with RedHat Ent6 does not have several v9 specific features we need).
>>
>> Did you confirm that there is a native spinlocks implementation and it is
>> being used? - I believe so as no errors or warnings logged during the build.
>> Is there a simple way to check whether spin-locks are running native?
>>
>> I've started looking at several articles covering pgbench and running some
>> initial tests, so I plan to start a new thread on pg-performance in the next
>> day or so.
>>
>> Thanks for the advice so far - Appreciated :-)
> I can't remember off the top of my head if configure forces you to
> specifically unset spinlocks to get through a build on a non-hardware
> spinlock platform. Point being: the interesting stuff happens during
> configure, not build.
>
> Check the contents of src/include/pg_config.h and look for this line:
> #define HAVE_SPINLOCKS 1
>
> to see if you have hardware spinlocks.
>
> merlin
>
>
Confirm that #define HAVE_SPINLOCKS 1 is present and correct.

Will move any performance related issues I find onto the pg-performance
list.
Many thanks for all the help and advice so far :-)
Andrew