Re: Postgres with pthread

Hi,

On 2017-12-06 11:53:21 -0500, Tom Lane wrote:
> Konstantin Knizhnik <k(dot)knizhnik(at)postgrespro(dot)ru> writes:
> However, if I guess at which numbers are supposed to be what,
> it looks like even the best case is barely a 50% speedup.

"barely a 50% speedup" - Hah. I don't believe the numbers, but that'd be
huge.

> That would be worth pursuing if it were reasonably low-hanging
> fruit, but converting PG to threads seems very far from being that.

I don't think immediate performance gains are the interesting part about
using threads. It's rather what their absence adds a lot in existing /
submitted code complexity, and makes some very commonly requested
features a lot harder to implement:

- we've a lot of duplicated infrastructure around dynamic shared
memory. dsm.c dsa.c, dshash.c etc. A lot of these, especially dsa.c,
are going to become a lot more complicated over time, just look at how
complicated good multi threaded allocators are.

- we're adding a lot of slowness to parallelism, just because we have
different memory layouts in different processes. Instead of just
passing pointers through queues, we put entire tuples in there. We
deal with dsm aware pointers.

- a lot of features have been a lot harder (parallelism!), and a lot of
frequently requested ones are so hard due to processes that they never
got off ground (in-core pooling, process reuse, parallel worker reuse)

- due to the statically sized shared memory a lot of our configuration
is pretty fundamentally PGC_POSTMASTER, even though that present a lot
of administrative problems.

...

> I think you've done us a very substantial service by pursuing
> this far enough to get some quantifiable performance results.
> But now that we have some results in hand, I think we're best
> off sticking with the architecture we've got.

I don't agree.

I'd personally expect that an immediate conversion would result in very
little speedup, a bunch of code deleted, a bunch of complexity
added. And it'd still be massively worthwhile, to keep medium to long
term complexity and feature viability in control.

Greetings,

Andres Freund