Re: Idle processes chewing up CPU?

Hi Craig, thanks for the analysis. If I attach a debugger on the runaway
child process, will this halt execution for all the other child processes
(ie. freeze the server)? And, can I attach Visual Studio C++ 2008, or is
there a recommended debugger for Windows debugging?

Given the reliability of the server in the past, I'd probably be expecting
an issue with OpenSSL instead, but with debugging attached I should be able
to say for sure.

Regards,
-Brendan

-----Original Message-----
From: Craig Ringer [mailto:craig(at)postnewspapers(dot)com(dot)au]
Sent: Wednesday, 19 August 2009 3:12 PM
To: Brendan Hill
Cc: pgsql-general(at)postgresql(dot)org; 'Tom Lane'
Subject: Re: [GENERAL] Idle processes chewing up CPU?

On 19/08/2009 12:31 PM, Brendan Hill wrote:
> Hi Craig/Tom,
>
> I've managed to trap the full stack trace this time

The common part of those traces is:

> ntdll.dll!KiFastSystemCallRet
> WS2_32.dll!WSARecv+0x65
> WSOCK32.dll!recv+0x31
> LIBEAY32.dll!BIO_sock_should_retry+0x57
> postgres.exe!my_sock_read+0x1b
> LIBEAY32.dll!BIO_read+0x6f
> SSLEAY32.dll!SSLv3_client_method+0x1ee1
> SSLEAY32.dll!SSLv3_client_method+0x22ea
> mswsock.dll!StartWsdpService+0x500
> SSLEAY32.dll!SSLv3_client_method+0x225a
> SSLEAY32.dll!SSLv3_client_method+0x2a15
> postgres.exe!pgwin32_waitforsinglesocket+0x1ed
> postgres.exe!secure_read+0x26
> postgres.exe!pq_recvbuf+0x71
> postgres.exe!pq_getbyte+0x15
> postgres.exe!SocketBackend+0x6
> postgres.exe!PostgresMain+0xbf8
> postgres.exe!BackendRun+0x200
> postgres.exe!SubPostmasterMain+0x21d
> postgres.exe!main+0x177
> postgres.exe!__tmainCRTStartup+0x10f
> kernel32.dll!ProcessIdToSessionId+0x209

Now, it's not possible to tell for sure from the traces alone whether
this part of the trace shows the same instances of the same function
calls, or whether there's a loop happening such that (eg)
pgwin32_waitforsinglesocket is being called over and over and over. To
find that out, you'd need to attach a debugger and set a breakpoint
somewhere suitable.

Personally, though, as a somewhat informed stab in the dark I suspect
that the above part of the call stack is actually entered once and not
left. I'd say that when Pg calls my_sock_read(...), resulting in a call
to recv(...) and from there a kernel system call, that's as far as it
goes. The system call never returns.

Why? I suspect you have a buggy network driver or faulty network card.
The unexpected interrupt hander being called in one of the stack tracces
certainly has to make you wonder.

> I'd appreciate any help diagnosing this problem - cutting off remote
access
> via SSL isn't the ideal solution.

I'd replace the NIC with one from a different manufacturer, at least
temporarily. I won't be shocked if the problem goes away.

--
Craig Ringer