Re: anole: assorted stability problems

On Sun, Jun 28, 2015 at 9:10 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Sun, Jun 28, 2015 at 7:27 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> I'd hoped that commit 1b468a131bd260c9041484f78b8580c7f232d580 would
>> resolve this, but nope, anole is still getting occasional stuck spinlocks:
>> http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=anole&dt=2015-06-28%2021%3A35%3A02
>
> That sucks. It was easy to see that the old fallback barrier
> implementation wasn't re-entrant, but this one should be. And now
> that I look at it again, doesn't the failure message indicate that's
> not the problem anyway?
>
> ! PANIC: stuck spinlock (c00000000d6f4140) detected at lwlock.c:816
> ! PANIC: stuck spinlock (c00000000d72f6e0) detected at lwlock.c:770
>
> That's just a straight-up SpinLockAcquire(), not a barrier call.
>
> The May 5th failure looked like this:
>
> ! FATAL: semop(id=0) failed: Result too large
>
> The May 1st failure seems to have died here:

Ugh, hit "send" too soon.

create temp table tc (id int primary key, aid int);
! server closed the connection unexpectedly
! This probably means the server terminated abnormally
! before or while processing the request.
! connection to server was lost

Both the "Result too large" and the lwlock.c stuck spinlock failures
have been repeated a number of times. I think the "result too large"
error from semop() is probably a clue - is it possible that we somehow
go into an infinite loop incrementing the semaphore? What else could
that error mean?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company