Re: Decrease MAX_BACKENDS to 2^16

On 2014-04-28 10:48:30 +0300, Heikki Linnakangas wrote:
> On 04/26/2014 09:27 PM, Andres Freund wrote:
> >I don't think we need to decide this without benchmarks proving the
> >benefits. I basically want to know whether somebody has an actual
> >usecase - even if I really, really, can't think of one - of setting
> >max_connections even remotely that high. If there's something
> >fundamental out there that'd make changing the limit impossible, doing
> >benchmarks wouldn't be worthwile.
>
> It doesn't seem unreasonable to have a database with tens of thousands of
> connections. Sure, performance will suffer, but if the connections sit idle
> most of the time so that the total load is low, who cares. Sure, you could
> use a connection pooler, but it's even better if you don't have to.

65k connections will be absolutely *disastrous* for performance because
of the big PGPROC et al. I *do* think we have to make live easier for
users here by supplying builtin pooling at some point, but that's just a
separate feature.

> If there are big gains to be had from limiting the number of connections,
> I'm not against it. For the purpose of shrinking BufferDesc though, I have
> feeling there might be other lower hanging fruit in there. For example,
> wait_backend_pid and freeNext are not used very often, so they could be
> moved elsewhere, to a separate array. And buf_id and the LWLock pointers
> could be calculated from the memory address of the struct.

The main reason I want to shrink it is that I want to make pin/unpin
buffer lockless and all solutions I can come up with for that require
flags to be in the same uint32 as the refcount. For performance
it'd be beneficial if usagecount also fits in there.

I agree that we can move a good part of BufferDesc into a separately
indexed array. io_in_progress_lock, freeNext, wait_backend_id are imo
good candidates.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services