Re: Scaling up PostgreSQL in Multiple CPU / Dual Core

On Thu, Mar 23, 2006 at 09:22:34PM -0500, Christopher Browne wrote:
> Martha Stewart called it a Good Thing when smarlowe(at)g2switchworks(dot)com (Scott Marlowe) wrote:
> > On Thu, 2006-03-23 at 10:43, Joshua D. Drake wrote:
> >> > Has someone been working on the problem of splitting a query into pieces
> >> > and running it on multiple CPUs / multiple machines? Yes. Bizgress has
> >> > done that.
> >>
> >> I believe that is limited to Bizgress MPP yes?
> >
> > Yep. I hope that someday it will be released to the postgresql global
> > dev group for inclusion. Or at least parts of it.
>
> Question: Does the Bizgress/MPP use threading for this concurrency?
> Or forking?
>
> If it does so via forking, that's more portable, and less dependent on
> specific complexities of threading implementations (which amounts to
> non-portability ;-)).
>
> Most times Jan comes to town, we spend a few minutes musing about the
> "splitting queries across threads" problem, and dismiss it again; if
> there's the beginning of a "split across processes," that's decidedly
> neat :-).

Correct me if I'm wrong, but there's no way to (reasonably) accomplish
that without having some dedicated extra processes laying around that
you can use to execute the queries, no? In other words, the cost of a
fork() during query execution would be too prohibitive...

FWIW, DB2 executes all queries in a dedicated set of processes. The
process handling the connection from the client will pass a query
request off to one of the executor processes. I can't remember which
process actually plans the query, but I know that the executor runs it.
--
Jim C. Nasby, Sr. Engineering Consultant jnasby(at)pervasive(dot)com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461