Postmaster dies with many child processes (spinlock/semget failed)

Hi,

I sent the following message to the pgsql-general
list on the 24th but haven't received any answers
from PostgreSQL developers, only from other people
who are experiencing the same problems.

I would say the errors I am describing are quite
serious and I was wondering whether there was any
chance of them being addressed in the forthcoming
6.5 release.

The problem is very easy to reproduce - here are
the necessary steps:

1. Install PostgreSQL 6.4.2
2. Install Perl 5.005_02
3. Install Perl modules: DBI 1.06; DBD-Pg 0.90; ApacheDBI-0.81
3. Download apache 1.3.4
4. Download mod_perl 1.17+ in same directory
5. Extract distributions
6. cd mod_perl-1.17
7. perl Makefile.PL EVERYTHING=1 && make && make test && make install
8. Set the following directives in Apache's httpd.conf:
MinSpareServers 100
MaxSpareServers 100
StartServers 100
MaxClients 100
9. PerlRequire /usr/local/apache/conf/startup.pl where
startup.pl contains:
use Apache::Registry ();
use Apache::DBI ();
Apache::DBI->connect_on_init("DBI:Pg:dbname=template1", "", "");
1;
10. Start Apache: apachectl start

Note that this example makes use of no custom
application code and is using the template1
database.

Check Apache's error_log and you will see error
messages and eventually the postmaster will die
with something like:

FATAL: s_lock(28001065) at spin.c:125, stuck spinlock. Aborting.

The magic number seems to be 48. If I start 49
httpd/postgres processes everything falls apart
but if I start 48 everything is fine. I'm
running on FreeBSD 2.2.8 and I've increased
maxusers to 512 - no difference.

I'd appreciate some feedback from the guys who
are making PostgreSQL happen. Can these issues
be addressed? PostgreSQL is a great database but
this is a show stopper for people developing big
Web applications.

If you need any more information don't hesitate
to contact me.

Cheers.

Patrick

--

Sent to pgsql-general list on : January 24th 1999

Hi,

I've been doing some benchmarking with PostgreSQL
under mod_perl and I've been getting some rather
disturbing results. To achieve the maximum benefit
from persistent connections I am using a method
called 'connect_on_init' that comes with a Perl
module called Apache::DBI. Using this method,
when the Web server is first started - each child
process establishes a persistent connection with
the database. When using PostgreSQL as the database,
this causes there to be as many 'postgres'
processes are there are 'httpd' processes
for a given database.

As part of my benchmarking I've been testing the
number of httpd processes that my server can
support. The machine is a 450 MHz PII/256 MB RAM.
As an excercise I tried to start 100 httpd
processes. Doing this consistently results in the
following PostgreSQL errors and the backend usually
dies:

IpcSemaphoreCreate: semget failed (No space left on device) key=5432017, num=16, permission=600
NOTICE: Message from PostgreSQL backend:
The Postmaster has informed me that some other backend died abnormally and possibly corrupted shared memory.
I have rolled back the current transaction and am going to terminate your database system connection and exit.
Please reconnect to the database system and repeat your query.

FATAL: s_lock(28001065) at spin.c:125, stuck spinlock. Aborting.

Note that the 'no space left on device' is
misleading as there is a minimum of 400 MB
available on each file-system on the server.

This is obviously bad news, especially as we are
hoping to develop some fairly large-scale
applications with PostgreSQL. Note that this
happens when connecting to a single database.
We were hoping to connect to several databases
from each httpd process!!

The frustrating thing is we have the resources.
If I only start 30 processes (which seems to be
the approximate limit) there is about 100 MB
of RAM that is not being used.

Are there any configuration values that control
the number of postgres processes? Do you have
any idea why this is happening?

Is anyone else using Apache/mod_perl and PostgreSQL
successfully in a demanding environment?

Any help would be greatly appreciated.

Cheers.

Patrick

--

#===============================#
\ KAN Design & Publishing Ltd /
/ T: +44 (0)1223 511134 \
\ F: +44 (0)1223 571968 /
/ E: mailto:patrick(at)kan(dot)co(dot)uk \
\ W: http://www.kan.co.uk /
#===============================#