Re: Multithreaded query onto 4 postgresql instances

From: Alessandro Candini <candini(at)meeo(dot)it>
To: Alban Hertroys <dalroi(at)solfertje(dot)student(dot)utwente(dot)nl>
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: Multithreaded query onto 4 postgresql instances
Date: 2011-02-16 08:54:59
Message-ID: 4D5B90E3.3080009@meeo.it
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Il 15/02/2011 19:32, Alban Hertroys ha scritto:
> On 15 Feb 2011, at 9:32, Alessandro Candini wrote:
>
>>> Is that a single query on that one DB compared to 4 queries on 4 DB's? How does a single DB with 4 parallel queries perform? I'd expect that to win from 4 DB's, due to the overhead those extra DB instances are generating.
>> Maybe my configuration and test is not clear
> It is clear. I gave you another suggestion for something to try instead.
>
>> Splitted instances:
>> 600 millions of records in total splitted into 4 postgresql instances (port 5433, 5434, 5435, 5436), let's say more or less:
>>
>> 5433 ---> 150 millions of records
>> 5434 ---> 150 millions of records
>> 5435 ---> 150 millions of records
>> 5436 ---> 150 millions of records
> Try the above on a single DB using 4 threads. It will very probably perform better.
> To use your example:
> 5432 ---> 150 million records
> 5432 ---> 150 million records
> 5432 ---> 150 million records
> 5432 ---> 150 million records
>

Excuse me but query must to be performed on the whole db...with your
approach, how to merge results of every query in one single response?

>>> Did you read all the way to section 35.9.10? That explains how to create SRF's like yours, including examples. If the step from simple functions to SRF's is too large for you, create a few sample-functions to learn how the intermediary steps work. We can't upload the info to your brain, after all - you'll have to do the learning part by yourself.
>>>
>> Do you think is a good idea continue using libpq or should I abandon them and focus on postgresql documentation examples?
> To learn how to write functions like these? Best to use a simple case so it's clear what's going on. If you try to alter already complicated code for that purpose you'll probably just get more confused.
>
> Alban Hertroys
>
> --
> Screwing up is an excellent way to attach something to the ceiling.
>
>
> !DSPAM:1234,4d5ac6b211738438191676!
>
>

--
Alessandro Candini

MEEO S.r.l.
Via Saragat 9
I-44122 Ferrara, Italy
Tel: +39 0532 1861501
Fax: +39 0532 1861637
http://www.meeo.it

========================================
"ATTENZIONE:le informazioni contenute in questo messaggio sono
da considerarsi confidenziali ed il loro utilizzo è riservato unicamente
al destinatario sopra indicato. Chi dovesse ricevere questo messaggio
per errore è tenuto ad informare il mittente ed a rimuoverlo
definitivamente da ogni supporto elettronico o cartaceo."

"WARNING:This message contains confidential and/or proprietary
information which may be subject to privilege or immunity and which
is intended for use of its addressee only. Should you receive this
message in error, you are kindly requested to inform the sender and
to definitively remove it from any paper or electronic format."

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Adarsh Sharma 2011-02-16 09:54:45 pg_dump with select output
Previous Message AI Rumman 2011-02-16 08:36:03 why does the toast table exist?