Re: neither CPU nor IO bound, but throttled performance

On Thu, Feb 21, 2019 at 12:34 AM Gunther <raj(at)gusw(dot)net> wrote:

> Hi, I have an Amazon Linux based Postgresql 11 server here on a t2.medium
> EC2 instance.
>
> It is serving 24 worker processes that read jobs from a queue (thanks to
> SELECT ... FOR UPDATE SKIP LOCKED!) and do jobs some of which are reading
> and writing business data to the database, others are only reading, and
> some don't hit the business data at all, only the queue.
>
> Everything flows quite nicely. Except, I don't understand why I can't max
> out the CPU or the IO, instead, IO is almost negligible yet the CPU is at
> 30% hardly hitting 50%.
>
> Here I give you a view of top:
>
> top - 23:17:09 up 45 days, 2:07, 4 users, load average: 20.32, 18.92, 13.80
> Tasks: 338 total, 24 running, 111 sleeping, 0 stopped, 0 zombie
> %Cpu(s): 28.7 us, 2.5 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 0.0 si, 68.7 st
>
>
If I read that right, it's about 70% "steal". The description for this is
"the percentage of time spent in involuntary wait by the virtual CPU or
CPUs while the hypervisor was servicing another virtual processor.".

So basically, the CPU is spent dealing with other peoples VMs on the same
hardware. Welcome to the multi-tenant cloud.

In particular, I believe T series instances get a limited number of CPU
"credits" per hours. My guess is you've hit this limit and are thus being
throttled. T series are not intended for persistent workloads. Either way,
this is probably a question better asked at the Amazon EC2 forums rather
than PostgreSQL as you'll find more people who know the EC2 interactions
there.

--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/>
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>