Re: Parallel Execution of Query

From: Shmagi Kavtaradze <kavtaradze(dot)s(at)gmail(dot)com>
To: James Keener <jim(at)jimkeener(dot)com>, pgsql-novice(at)postgresql(dot)org
Subject: Re: Parallel Execution of Query
Date: 2015-11-30 21:10:05
Message-ID: CAHY6maxyP2uPDdXHpb0dXoSqyiab8xuSzsPUoocmcJF+UZb18A@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-novice

Thanks for answer. What about async? can you tell in details?

On Mon, Nov 30, 2015 at 9:56 PM, James Keener <jim(at)jimkeener(dot)com> wrote:

> While parallel queries is an area of ongoing work, I do not believe that
> a single connection can currently use more than a single core. You
> could experiment with turning your 4000x4000 comparisons into 4 queries
> doing 2000x2000 comparison (assuming you have 4 cores available to the
> VM, the more cores, the more cores you could split it up) (just remember
> to use async for each of those so you don't execute them in serial!).
>
> Jim
>
> On 11/30/2015 12:00 PM, Shmagi Kavtaradze wrote:
> > I am doing similarity matching (cosine, euclidean). If I have 4000
> > entries in a table, the number of comparisons will be 16M. I am running
> > postgres on a virtual machine, so it takes 20-25 minutes to run the
> > query or the system crashes. Can I run the query in parallel? I heard
> > there are tools like PL/Proxy and pgpool, can I use them to create
> > several databases on the same machine and run query in parallel?
>
>

In response to

Responses

Browse pgsql-novice by date

  From Date Subject
Next Message James Keener 2015-11-30 21:19:46 Re: Parallel Execution of Query
Previous Message James Keener 2015-11-30 20:56:41 Re: Parallel Execution of Query