Re: One process per session lack of sharing

On Sun, Jul 17, 2016 at 10:00 PM, Jan Wieck <jan(at)wi3ck(dot)info> wrote:
>> I admit that it is risky, but I think there are things that could be
>> done to limit the risk. I don't believe we can indefinitely continue
>> to ignore the potential performance benefits of making a switch like
>> this. Breaking a thirty-year old code base irretrievably would be
>> sad, but letting it fade into irrelevance because we're not willing to
>> make the architecture changes that are needed to remain relevant would
>> be sad, too
>
> I have to agree with Robert on that one. We have been "thinking" about
> multi-threading some 16 years ago already. We were aware of the dangers
> and yet at least considered doing it some day for things like a parallel
> executor. And that would probably be our best bang for the buck still.
>
> The risks of jamming all sessions into a single, multi-threaded process are
> huge. Think of snapshot visibility together with catalog cache
> invalidations.
> I'd say no to that one as a first step.
>
> But multi-threading the executor or even certain utility commands at first
> should not be rejected purely on the notion that "we don't have
> multithreading
> today."

I think the risk profile is exactly the opposite of what you are
suggesting here. If we provide an option to compile the server with
all global variables converted to thread-local variables, there's
really not a whole lot that can break, AFAICS. We'll technically be
multi-threaded but the code need not know or care about the other
threads; only in the event of a memory clobber can they affect each
other.

On the other hand, if we start trying to create multiple threads per
processes just for certain purposes - be it the executor or certain
utility commands - any C code that is reachable within the secondary
threads needs to have specific provisions for thread-safety. That
would create all kinds of problems, no doubt.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company