Re: Postgresql server gets stuck at low load

On Tue, Jun 09, 2020 at 01:54:21PM +0200, Krzysztof Olszewski wrote:
> I had hugepage's off and on, problems still occurs,
> thanx for "perf top" suggestion,

> pt., 5 cze 2020 o 13:38 Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com> napisał(a):
> > pá 5. 6. 2020 v 12:07 odesílatel Krzysztof Olszewski <kolszew73(at)gmail(dot)com> napsal:
> >
> >> I have problem with one of my Postgres production server. Server works
> >> fine almost always, but sometimes without any increase of transactions or
> >> statements amount, machine gets stuck. Cores goes up to 100%, load up to
> >> 160%. When it happens then there are problems with connect to database and
> >> even it will succeed, simple queries works several seconds instead of
> >> milliseconds.Problem sometimes stops after a period a time (e.g. 35 min),
> >> sometimes we must restart Postgres, Linux, or even KVM (which exists as
> >> virtualization host).
> >>
> >> My hardware
> >> 56 cores (Intel Core Processor (Skylake, IBRS))
> >> 400 GB RAM
> >> RAID10 with about 40k IOPS
> >>
> >> shared_buffers 64000 MB
> >>
> >> In correlation with stucks i see informations in kernel log about
> >> NMI watchdog: BUG: soft lockup - CPU#25 stuck for 23s! [postmaster:33935]
> >
> > https://www.postgresql.org/message-id/CAHyXU0yAsVxoab2PcyoCuPjqymtnaE93v7bN4ctv2aNi92fefA%40mail.gmail.com
> >
> > Can be interesting a reply on Merlin's question from mail/.
> >
> > cat /sys/kernel/mm/redhat_transparent_hugepage/enabled
> > cat /sys/kernel/mm/redhat_transparent_hugepage/defrag

try this:
echo 2 |sudo /sys/kernel/mm/ksm/run

https://www.postgresql.org/message-id/20170718180152.GE17566%40telsasoft.com

--
Justin