Re: dynamic background workers

On 2013-06-20 11:29:27 -0400, Robert Haas wrote:
> > Do you mean pre-forking and connecting to a specific database? Or really
> > just the forking?
>
> I've considered both at various times, although in this context I was
> mostly thinking about just the forking. Pre-connecting to a specific
> database would save an unknown but possibly significant amount of
> additional latency. Against that, it's more complex (because we've
> got to track which preforked workers are associated with which
> databases) and there's some cost to guessing wrong (because then we're
> keeping workers around that we can't use, or maybe even having to turn
> around and kill them to make slots for the workers we actually need).
> I suspect we'll want to pursue the idea at some point but it's not
> near the top of my list.

Just as a datapoint, if you benchmark the numbers of forks that can be
performed by a single process (i.e. postmaster) the number is easily in
the 10s of thousands. Now forking that much has some scalability
implications inside the kernel, but still.
I'd be surprised if the actual fork is more than 5-10% of the current
cost of starting a new backend.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services