From: | Tim Holloway <mtsinc(at)southeast(dot)net> |
---|---|
To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
Cc: | pgsql-hackers <pgsql-hackers(at)postgreSQL(dot)org> |
Subject: | Re: [HACKERS] Thread-safe queueing? |
Date: | 1999-11-14 00:58:14 |
Message-ID: | 382E0926.2C3F536E@southeast.net |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Tom Lane wrote:
>
> Tim Holloway <mtsinc(at)southeast(dot)net> writes:
> > I need to create a cross-process producer/consumer data queue
> > (e.g. singly-linked list). That is - Processes A, B, and C add nodes
> > to a controlled list and process D removes them. Not sure if the
> > creation of the nodes would be best done by the producers or
> > consumers, but destruction would have to be done by the consumer, as
> > the producers don't wait for processing. For optimal results, the
> > consumer process should sleep until item(s) are added to its queue.
>
> > Query: within the existing backend framework, what's the best way to
> > accomplish this?
>
> More context, please. What are you trying to accomplish? Is this
> really a communication path between backends (and if so, what backend
> code needs it?), or are you trying to set up a queue between SQL
> clients? How much data might need to be in the queue at one time?
>
> regards, tom lane
>
This is for the logging subsystem I'm developing. The backends call pg_log(),
which is like elog(), except that the message is a resource ID + any parameters
in order to support locales and custom message formatting. These ID+parameter
packets are then pipelined down to the logging channels via the log engine to
be formatted and output according to rules in the configuration file.
I *think* that the log engine should be a distinct process. I'm not sure I can
trust the output not to come out sliced and diced if each backend can run the engine
directly -- and for that matter, I see problems if the engine is reconfigured on the
fly owing to the need for each backend to replicate the configuration process (among
other things). The basic singly-linked list component is all I need to handle the
FIFO, but obviously I need guards to preserve its integrity. As to the amount of data
involved, I sincerely hope the queue would stay pretty shallow!
I have the configuration parser and logging engine operational, so the last
significant hurdle is making sure that A) the data to be logged is
accessable/addressable by the engine, and B) that the process runs in the
proper sequence. A description of what it all will look like is now online at http://postgres.mousetech.com/index.html
(with apologies for the ugly formatting).
Thanks,
TIm Holloway
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 1999-11-14 02:28:54 | Re: [HACKERS] Thread-safe queueing? |
Previous Message | Cary O'Brien | 1999-11-13 22:23:15 | Compression in LO and other fields |