Re: Scalability in postgres

On Fri, Jun 5, 2009 at 12:33 AM, <david(at)lang(dot)hm> wrote:
> On Fri, 5 Jun 2009, Greg Smith wrote:
>
>> On Thu, 4 Jun 2009, Robert Haas wrote:
>>
>>> That's because this thread has altogether too much theory and
>>> altogether too little gprof.
>>
>> But running benchmarks and profiling is actual work; that's so much less
>> fun than just speculating about what's going on!
>>
>> This thread reminds me of Jignesh's "Proposal of tunable fix for
>> scalability of 8.4" thread from March, except with only a fraction of the
>> real-world detail.  There are multiple high-profile locks causing
>> scalability concerns at quadruple digit high user counts in the PostgreSQL
>> code base, finding them is easy.  Shoot, I know exactly where a couple are,
>> and I didn't have to think about it at all--just talked with Jignesh a
>> couple of times, led me right to them.  Fixing them without causing
>> regressions in low client count cases, now that's the hard part.  No amount
>> of theoretical discussion advances that any until you're at least staring at
>> a very specific locking problem you've already characterized extensively via
>> profiling.  And even then, profiling trumps theory every time.  This is why
>> I stay out of these discussions and work on boring benchmark tools instead.
>
> actually, as I see it we are a step before that.
>
> it seems that people are arguing that there is no need to look for and fix
> this sort of thing, on the basis that anyone who trips over these problems
> is doing something wrong to start with and needs to change the behavior of
> their app.

I have a slightly different take on that. I don't think there's
actually resistance to improving this situation if someone (or some
group of people) comes up with a good proposal for doing it and writes
a patch and tests it and shows that it helps that case without hurting
other cases that people care about. And there is clearly great
willingness to tell people what to do until that happens: use
connection pooling. But if you come back and say, well, I shouldn't
have to use connection pooling because it should work without
connection pooling, well, OK, but...

...Robert