Re: postgresql + apache under heavy load

On Wed, 21 Jan 2004, Alex Madon wrote:

> Hello Scott,
> Thank you for your answer.
>
> >I'm not familiar with DBX. Is that connection pooling or what?
> >
> >
> I could not find this information, sorry.
>
> >Are you SURE all your memory is in use? What exactly does top say about
> >things like cached and buff memory (I'm assuming you're on linux, any
> >differences in top on another OS would be minor.) If the kernel still
> >shows a fair bit of cached and buff memory, your memory is not getting all
> >used up.
> >
> Well my xosview show that caching begin at a concurrency of 40.
> At 80 my cache begins to be filled completely, so machine having big
> problems.

I think you're confusing what I meant. Caching is good. Swapping is bad.
Having a large amount of cache is a good thing. It means the OS is
caching all your data in memory for faster access.

> >>If I increases to
> >>ab -n 1000 -c 100 http://localsite/testscript
> >>I get this memory problem.
> >>
> >>
> >
> >Where's the break point? Just wondering. Does it show up at 20, 40, 60,
> >80, or only at 100? If so, that's really not bad.
> >
> Here is some results (I kept -n 100 an just vraied the -c option)
> --c 1
> Failed requests: 0
> Time per request: 322.096 [ms] (mean, across all concurrent requests)
>
> -c 2
> Failed requests: 0
> Time per request: 374.220 [ms] (mean, across all concurrent requests)
>
> -c 10
> Failed requests: 68
> (Connect: 0, Length: 68, Exceptions: 0)
> Time per request: 314.779 [ms] (mean, across all concurrent requests)

OK, there's a problem, you're getting failed requests at -c 10, which
means you likely have postgresql configured in the wrong
direction. configure postgresql to use more memory (sort_mem can be set
to about 8 megs without a lot of issues on most boxes, going higher may
use up all your memory in certain situations (high concurrency)).

> For a higher concurrency, the cache is completly filled, and have to
> reboot the machine.

No, you should NEVER have to reboot a unix box. period. filled cache,
again, is a GOOD THING. not bad.

> (I didn't leave the system caching forever, just press to reboot
> button)... could be interesting to wait to see if the systems recovers
> after a while

Yes, please do. Also, show us a save of top while under load.

I'm betting your machine has plenty of memory, and is not using it
effectively, due to postgresql being too conservatively configured.