Re: [mux@FreeBSD.org: Re: Anyone interested in improving postgresql scaling?]

On Wed, Apr 11, 2007 at 01:03:50AM -0400, Tom Lane wrote:
> Kris Kennaway <kris(at)obsecurity(dot)org> writes:
> >>>>> I think the high number of setproctitle() calls are more problematic
> >>>>> to us at the moment, Kris can comment on that.
>
> > Since we've basically had it handed to us that calling setproctitle()
> > thousands of times per second is something that real applications now
> > do, we're pretty much forced to work on making it cheaper.
> > ...
> > However this won't help all the existing systems out there (including
> > other affected OSes), so it would be great if you guys could meet us
> > half way and find a way to make postgresql rate-limit these calls by
> > default to some suitable compromise rate, like once/second or
> > whatever.
>
> Well, the thing is, we've pretty much had it handed to us that
> current-command indicators that aren't up to date are not very useful.
> So rate-limited updates strike me as a useless compromise. We have
> the "real" solution (status advertised in PG's shared memory) already,
> so the question in my mind is just how fast DBAs will wish to transition
> to looking at "select * from pg_stat_activity" instead of looking at
> "ps auxww".

I don't get your argument - ps auxww is never going to be 100%
up-to-date because during the time the command is running the status
may change. So we already know that stats being a fraction of a
second out of date are acceptable to users, because that's what may
happen when you run ps in the present model. So you can use this to
get away with limiting updates to e.g. 10/second and in practise no
users will notice the difference.

Updating thousands of times a second just on the off chance that an
admin may one day run ps is completely inefficient (and has a huge
overhead on non-Linux systems, so it's demonstrably not a sensible way
to do things), and to the extent that there is a problem to be solved
it isn't even really solving it anyway.

If there really are users who find 10 proctitle updates/second an
unacceptably low update rate, then tune for the default case and
provide an option to allow them to override the rate limit to whatever
update rate they find appropriate.

Kris