Re: [HACKERS] posmaster failed under high load

Well,
just run test with 6.5 cvs and it looks much stable.
I run ./http_load -rate 20 -verbose -fetches 80 TEST
(notice, test is much stronger than in previous post) and got results:
81 fetches, 393 max parallel, 809028 bytes, in 24 seconds
9988 mean bytes/connection
3.375 fetches/sec, 33709.5 bytes/sec

My machine was very-very load during this test - I saw peak
load about 65, a lot of swapping but test completes and system
after 20 minutes of swapping remains usable. I still saw many
postmasters (not postgres) processes running but after about
30-40 minutes they gone. Actually pstree -a now shows

|-postmaster -i -B 1024 -S -D/usr/local/pgsql/data/ -o -Fe
| |-(postmaster)
| `-postmaster

Is this a normal behaivour for postmaster ?
I thought there is must be only one postmaster which forks postgres
processes for every connection. Anyway, system is usable,
postmaster survives and continue working ! 6.5 in this respect is much
stable. I run postmaster with -B 1024 option. This test I run under
Linux 2.2.7, so tomorrow I'll test on my production server which
runs Linux 2.0.36, SMP, Dual PPRO, 256 Mb Ram. As I wrote 6.4.2 fails
under high load, so I'll test 6.5 cvs to be sure what's is critical
kernel or postgres version.

Regards,

Oleg

On Wed, 5 May 1999, Tatsuo Ishii wrote:

> Date: Wed, 05 May 1999 00:02:44 +0900
> From: Tatsuo Ishii <t-ishii(at)sra(dot)co(dot)jp>
> To: Oleg Bartunov <oleg(at)sai(dot)msu(dot)su>
> Cc: Tatsuo Ishii <t-ishii(at)sra(dot)co(dot)jp>, hackers(at)postgreSQL(dot)org
> Subject: Re: [HACKERS] posmaster failed under high load
>
> > > I don't think so unless you increased the shared buffer size using -B
> > > option. Stock 6.4.2 is very buggy with the shared memory
> > > usage. Probably it's the cause. Try Tom Lane's fix or 6.5b. I have
> > > tested 6.5b with 128 backends running and it seems very stable.
> >
> > Yes, I used 6.4.2 + LIMIT patch, I'll try 6.5 from cvs
>
> You need Tom Lane's share mem fix patch if you use 6.4.2. 6.5 has the
> fix.
>
> > I run postmaster with -B 1024 option - is this too much ?
>
> No. -B 1024 means 8MB shared mem that should be ok on x86/Linux (I
> think most x86 based Linux allow 32MB shared mem).
>
> > Thanks a lot, I got several times a problem with file descriptors,
> > it looks like every backend opens abot 90 files. I'll try your
> > hints.
>
> But be carefull lower # of file descriptors per backend might cause
> lower performance because of the file opening overhead. So you should
> increase the file table entries in the system first.
>
> >Why not add your experience how to work with postgres under high
> > load to Linux specific FAQ ?
>
> I'm not good at English, that is the reason:-)
>
> BTW, FreeBSD box has more serious problems than Linux since the
> default kernel has lower limit of file descriptors (~700). This should
> be noted somewhere in the docs too.
> ---
> Tatsuo Ishii
>

_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83