Fwd: Database Stalls

On Mon, Jan 30, 2023 at 4:32 PM Mok <gurmokh(at)gmail(dot)com> wrote:

> Hi,
>
> Unfortunately there is no pg_stat_activity data available as we are
> unaware of the issue until it has already happened.
>
> The version we are on is 12.11.
>
> I don't think it is due to locks as there are none in the logs. Vacuums
> are logged also and none occur before or after this event. Checkpoint
> timeout is set to 1 hour and these events do not coincide with checkpoints.
>
> Gurmokh
>
>>
>>
Have you eliminated network issues? I have seen what looks like a database
stalling to end up actually being the network packets taking a side trip to
halfway around the world for a while. Or DNS lookups suddenly taking a
really long time.

The next most likely thing is disk i/o. Do you have huge corresponding
disk i/o spikes or does it drop completely to zero (which is also bad -
especially if you are on a SAN and you can't get any packets out on that
network). You'll have to look at your disks via OS tools to see.

Do you have any hardware faults? Errors on a hardware bus? Overheating?
I used to have a system that would freeze up entirely due to a problem with
a serial port that we had a console attached to - it was sending a low
level interrupt. Sometimes it would recover mysteriously if someone hit
the carriage return a couple times. Ie, is it _really_ the database that
is locking up, or is it your hardware?