| From: | James Keener <jim(at)jimkeener(dot)com> |
|---|---|
| To: | Shmagi Kavtaradze <kavtaradze(dot)s(at)gmail(dot)com>, pgsql-novice(at)postgresql(dot)org |
| Subject: | Re: Parallel Execution of Query |
| Date: | 2015-11-30 20:56:41 |
| Message-ID: | 565CB809.4060700@jimkeener.com |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-novice |
While parallel queries is an area of ongoing work, I do not believe that
a single connection can currently use more than a single core. You
could experiment with turning your 4000x4000 comparisons into 4 queries
doing 2000x2000 comparison (assuming you have 4 cores available to the
VM, the more cores, the more cores you could split it up) (just remember
to use async for each of those so you don't execute them in serial!).
Jim
On 11/30/2015 12:00 PM, Shmagi Kavtaradze wrote:
> I am doing similarity matching (cosine, euclidean). If I have 4000
> entries in a table, the number of comparisons will be 16M. I am running
> postgres on a virtual machine, so it takes 20-25 minutes to run the
> query or the system crashes. Can I run the query in parallel? I heard
> there are tools like PL/Proxy and pgpool, can I use them to create
> several databases on the same machine and run query in parallel?
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Shmagi Kavtaradze | 2015-11-30 21:10:05 | Re: Parallel Execution of Query |
| Previous Message | Shmagi Kavtaradze | 2015-11-30 17:00:40 | Parallel Execution of Query |