Re: neither CPU nor IO bound, but throttled performance

Thank you Magnus. 68% steal. Indeed. You probably hit the target. Yes.

That explains the discrepancy. I need to watch and understand that CPU
credits issue.

regards,
-Gunther

On 2/21/2019 4:08, Magnus Hagander wrote:
> On Thu, Feb 21, 2019 at 12:34 AM Gunther <raj(at)gusw(dot)net
> <mailto:raj(at)gusw(dot)net>> wrote:
>
> Hi, I have an Amazon Linux based Postgresql 11 server here on a
> t2.medium EC2 instance.
>
> It is serving 24 worker processes that read jobs from a queue
> (thanks to SELECT ... FOR UPDATE SKIP LOCKED!) and do jobs some of
> which are reading and writing business data to the database,
> others are only reading, and some don't hit the business data at
> all, only the queue.
>
> Everything flows quite nicely. Except, I don't understand why I
> can't max out the CPU or the IO, instead, IO is almost negligible
> yet the CPU is at 30% hardly hitting 50%.
>
> Here I give you a view of top:
>
> top - 23:17:09 up 45 days, 2:07, 4 users, load average: 20.32,
> 18.92, 13.80 Tasks: 338 total, 24 running, 111 sleeping, 0
> stopped, 0 zombie %Cpu(s): 28.7 us, 2.5 sy, 0.0 ni, 0.0 id, 0.0
> wa, 0.0 hi, 0.0 si, 68.7 st
>
>
> If I read that right, it's about 70% "steal". The description for this
> is "the percentage of time spent in involuntary wait by the virtual
> CPU or CPUs while the hypervisor was servicing another virtual
> processor.".
>
> So basically, the CPU is spent dealing with other peoples VMs on the
> same hardware. Welcome to the multi-tenant cloud.
>
> In particular, I believe T series instances get a limited number of
> CPU "credits" per hours. My guess is you've hit this limit and are
> thus being throttled. T series are not intended for persistent
> workloads. Either way, this is probably a question better asked at the
> Amazon EC2 forums rather than PostgreSQL as you'll find more people
> who know the EC2 interactions there.
> --
>  Magnus Hagander
>  Me: https://www.hagander.net/ <http://www.hagander.net/>
>  Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>