From: | valgog <valgog(at)gmail(dot)com> |
---|---|
To: | pgsql-general(at)postgresql(dot)org |
Subject: | Re: Multithreaded queue in PgSQL |
Date: | 2008-07-15 09:00:57 |
Message-ID: | 677a6e52-a3ff-4c8a-8676-9078dca90e71@34g2000hsf.googlegroups.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
> Whole point is to have multiple services accessing same table and
> dividing the work, so locking with waiting for lock to be released is
> out of question.
>
We are doing the same (newsletter) and there is no problem to lock the
whole table for a short time with an advisory lock as the java id
fetching worker is locking the table (that does not lock the table for
reading or writing, it is only locking his java worker brothers that
are using the same advisory lock), fetches, let's say, 50 id's of
records marked as CREATED and changes their status to PROCESSING. Then
several workers are getting the id's and fetch the needed data from
the table independently and process and update them in parallel. We
have 3 java machines getting id's for 10 parallel workers and
everything works just fine.
Getting the IDs is much much faster usually then real processing.
From | Date | Subject | |
---|---|---|---|
Next Message | Ragnar | 2008-07-15 09:28:50 | Re: optimizer ignoring primary key and doing sequence scan |
Previous Message | Craig Ringer | 2008-07-15 08:45:13 | Re: 8.3.3 Complie issue |