Re: bg worker: patch 1 of 6 - permanent process

On 09/14/2010 08:41 PM, Robert Haas wrote:
> To avoid consuming system resources forever if they're not being used.

Well, what timeout would you choose. And how would you justify it
compared to the amounts of system resources consumed by an idle process
sitting there and waiting for a job?

I'm not against such a timeout, but so far I felt that unlimited would
be the best default.

> Well, presumably that would be fairly disastrous. I would think,
> though, that you would not have a min/max number of workers PER
> DATABASE, but an overall limit on the upper size of the total pool

That already exists (in addition to the other parameters).

> - I
> can't see any reason to limit the minimum size of the pool, but I
> might be missing something.

I tried to mimic what others do, for example apache pre-fork. Maybe it's
just another way of trying to keep the overall resource consumption at a
reasonable level.

The minimum is helpful to eliminate waits for backends starting up. Note
here that the coordinator can only request to fork one new bgworker at a
time. It then needs to wait until that new bgworker registers with the
coordinator, until it can request further bgworkers from the postmaster.
(That's due to the limitation in communication between the postmaster
and coordinator).

> Personally, my position is that if someone does something that is only
> a small improvement on its own but which has the potential to help
> with other things later, that's a perfectly legitimate patch and we
> should try to accept it. But if it's not a clear (even if small) win
> then the bar is a lot higher, at least in my book.

I don't think it's an improvement over the current autovacuum behavior.
Not intended to be one. But it certainly shouldn't hurt, either.

It only has the potential to help with other things, namely parallel
querying. And of course replication (Postgres-R). Or any other kind of
background job you come up with (where background means not requiring a
client connection).

Regards

Markus Wanner