Re: Pooling in Core WAS: Need help in performance tuning.

On Tue, Jul 27, 2010 at 9:56 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> On Tue, Jul 27, 2010 at 4:40 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> "Flushing them all" is not zero-cost; it's not too hard to believe that
>>> it could actually be slower than forking a clean new backend.
>
>> I'm not so sure I believe that.
>
> I'm not asserting it's true, just suggesting it's entirely possible.
> Other than the fork() cost itself and whatever authentication activity
> there might be, practically all the startup cost of a new backend can be
> seen as cache-populating activities.  You'd have to redo all of that,
> *plus* pay the costs of getting rid of the previous cache entries.
> Maybe the latter costs less than a fork(), or maybe not.  fork() is
> pretty cheap on modern Unixen.
>
>>> What's much worse, it's not zero-bug.
>
>> I think this is probably the biggest issue with the whole idea, and I
>> agree there would be some pain involved.
>
> Yeah, if it weren't for that I'd say "sure let's try it".  But I'm
> afraid we'd be introducing significant headaches in return for a gain
> that's quite speculative.

I agree that the gain is minimal of itself; after all, how often do
you need to switch databases, and what's the big deal if the
postmaster has to fork a new backend? Where I see it as a potentially
big win is when it comes to things like parallel query. I can't help
thinking that's going to be a lot less efficient if you're forever
forking new backends. Perhaps the point here is that you'd actually
sort of like to NOT flush all those caches unless it turns out that
you're switching databases - many installations probably operate with
essentially one big database, so chances are good that even if you
distributed connections / parallel queries to backends round-robin,
you'd potentially save quite a bit of overhead. Of course, for the
guy who has TWO big databases, you might hope to be a little smarter,
but that's another problem altogether.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company