Re: [HACKERS] posmaster failed under high load

Interesting, I just tried to load my home machine and got very
weird result:
76 ? S 0:00 postmaster -i -B 1024 -S -D/usr/local/pgsql/data/ -o -Fe
218 ? SW 0:02 (postmaster)
219 ? SW 0:02 (postmaster)
220 ? SW 0:01 (postmaster)
222 ? SW 0:02 (postmaster)
241 ? SW 0:02 (postmaster)
242 ? SW 0:02 (postmaster)
252 ? SW 0:01 (postmaster)
263 ? SW 0:01 (postmaster)
372 ? SW 0:00 (postmaster)
377 ? SW 0:00 (postmaster)
378 ? SW 0:00 (postmaster)
379 ? SW 0:00 (postmaster)
383 ? SW 0:00 (postmaster)
387 ? SW 0:00 (postmaster)
388 ? SW 0:00 (postmaster)
406 ? SW 0:00 (postmaster)

System is still in working conditions and psql could connects !
Postmasters seems dies with time, but after 15 minutes I still see
7 postmasters.
This is my scenario and setup:
P166, 64Mb RAM,
Linux 2.2.7,PostgreSQL 6.4.2 on i586-pc-linux-gnu, compiled by gcc egcs-2.91.5
( This is a stock 6.4.2 + LIMIT feature patch, Tatsuo suggests to use
Tom's shared memory patches but I didn't find them )

I have apache 1.3.6+mod_perl 1.19, Apache::DBI to open persistent
connection to database and Mason (http://www.masonhq.com) to
produce dynamical document which I retrieve using
http_load from http://www.acme.com/jef/ which I found is quite
useful for testing because it doesn't big down a client machine
(in my case client and server are on the same machine)
So, I run
./http_load -verbose -checksum -rate 25 -fetches 40 TEST,
where TEST is a file with URL to document.
If interesting here are benchmarks:
40 fetches, 117 max parallel, 403192 bytes, in 9 seconds
10079.8 mean bytes/connection
4.44444 fetches/sec, 44799.1 bytes/sec
38 bad checksums

I'm going to test 6.5 cvs.

Regards,

Oleg

On Wed, 5 May 1999, Tatsuo Ishii wrote:

> Date: Wed, 05 May 1999 00:02:44 +0900
> From: Tatsuo Ishii <t-ishii(at)sra(dot)co(dot)jp>
> To: Oleg Bartunov <oleg(at)sai(dot)msu(dot)su>
> Cc: Tatsuo Ishii <t-ishii(at)sra(dot)co(dot)jp>, hackers(at)postgreSQL(dot)org
> Subject: Re: [HACKERS] posmaster failed under high load
>
> > > I don't think so unless you increased the shared buffer size using -B
> > > option. Stock 6.4.2 is very buggy with the shared memory
> > > usage. Probably it's the cause. Try Tom Lane's fix or 6.5b. I have
> > > tested 6.5b with 128 backends running and it seems very stable.
> >
> > Yes, I used 6.4.2 + LIMIT patch, I'll try 6.5 from cvs
>
> You need Tom Lane's share mem fix patch if you use 6.4.2. 6.5 has the
> fix.
>
> > I run postmaster with -B 1024 option - is this too much ?
>
> No. -B 1024 means 8MB shared mem that should be ok on x86/Linux (I
> think most x86 based Linux allow 32MB shared mem).
>
> > Thanks a lot, I got several times a problem with file descriptors,
> > it looks like every backend opens abot 90 files. I'll try your
> > hints.
>
> But be carefull lower # of file descriptors per backend might cause
> lower performance because of the file opening overhead. So you should
> increase the file table entries in the system first.
>
> >Why not add your experience how to work with postgres under high
> > load to Linux specific FAQ ?
>
> I'm not good at English, that is the reason:-)
>
> BTW, FreeBSD box has more serious problems than Linux since the
> default kernel has lower limit of file descriptors (~700). This should
> be noted somewhere in the docs too.
> ---
> Tatsuo Ishii
>

_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83