Re: 100% cpu usage on some postmaster processes kill the complete database

From: Paul Dunkler <paul(dot)dunkler(at)xyrality(dot)com>
To: Richard Huxton <dev(at)archonet(dot)com>
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: 100% cpu usage on some postmaster processes kill the complete database
Date: 2012-03-01 19:41:25
Message-ID: 18DC6B22-3FEA-4655-94EE-DB3485F6E402@xyrality.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

I did that now - and analyzed the situation a bit. There are only queries running which will process very fast under high load (only index scans, very low rates of sequential scans). I found a remarkable number of Insert statements...

And sometimes when that happens, the CPU Utilization is going up to nearby 100% too and 98% is system usage...

Am 01.03.2012 um 18:02 schrieb Richard Huxton:

> On 01/03/12 16:41, Paul Dunkler wrote:
>> Hi List,
>>
>> we are currently running a rather large postgresql-installation with approximately 4k Transactions and 50k index scans per second.
>>
>> In the last days on some times of the day (irregular - 3-4 times a day), some of the postmaster processes are running with 100% cpu usage. That leads to a totally breakdown of the query execution. We see tons of statements which are correctly automatically aborted by our statement_timeout set to 15 seconds. I tried to search, but do not really recognize what the problem could be there...
>>
>> Some things i have checked:
>> - We are not running any bulk jobs or maintenance scripts at this time
>> - No system errors in any logs during that slowdowns
>> - I/O Performance seems fine. No high IO Wait amount... But IO Write totally drops in that times because it seems that no postgres process can perform any update
>>
>> I just installed a script, which prints me out the top and ps axf information for facing out the problem. I will post a snippet of the top here:
>
> Combine that with this:
> SELECT * FROM pg_stat_activity;
>
> That will let you line up pids from top with active queries.
>
> --
> Richard Huxton
> Archonet Ltd
>
> --
> Sent via pgsql-general mailing list (pgsql-general(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general

--
Mit freundlichen Grüßen

Paul Dunkler

-----------------------------------------------------
XYRALITY GmbH • Lerchenstraße 28a • 22767 Hamburg
Paul Dunkler • Softwareentwickler
Mail: paul(dot)dunkler(at)xyrality(dot)com
Tel: +49 (0) 40 23 51 78 97
Mobil: +49 (0) 151 252 228 42
Fax: +49 (0) 40 23 51 78 98
Web: http://www.xyrality.com/
Registergericht: Hamburg HRB 115332
Geschäftsführer: Sven Ossenbrüggen & Alexander Spohr
-----------------------------------------------------

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Richard Huxton 2012-03-01 20:16:12 Re: 100% cpu usage on some postmaster processes kill the complete database
Previous Message John R Pierce 2012-03-01 18:07:04 Re: Ident authentication failed for user