Re: Non-blocking communication between a frontend and a backend (pqcomm)

Hi,

Thanks for the comment!

On Tue, Jul 28, 2009 at 12:27 AM, Tom Lane<tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Well, actually, this description perfectly illustrates my basic
> complaint: the patch breaks the API abstraction provided by pqcomm.c.
> Callers are encouraged/forced to deal with the next layer down, and to
> the extent that pqcomm.c does anything useful, callers have to allow for
> that too.
>
> As far as the read side of pq_wait goes, it should probably be more
> like
> * return true if any bytes are in the buffer
> * else try to read some bytes into the buffer, without blocking
> * now return true or false depending on whether bytes are in
> the buffer.
> (Or perhaps instead of true/false, return the count of available bytes.)

Seems good. But when we wait for 4 bytes and there is 1 byte in the buffer,
the above pq_wait always returns immediately, and we cannot get 4 bytes.
So, I think that we should make pq_wait monitor only the socket. And, we
should make the newly-introduced low-level I/O functions (such as
pq_getbufbytes) load some bytes into the buffer without blocking (i.e.,
pq_wait with no timeout is called by those functions), and return true/false
according to whether the specified bytes are in there. What is your opinion?

> Also, I suspect the API needs to make a distinction between "no more
> bytes available yet" and EOF (channel closed, so no more bytes ever
> will be available).

That's right.

> I'm not sure about the write side. The patch isn't really addressing
> blocking-on-write, except in the offhand way of having a forWrite option
> in pq_wait, but that doesn't seem too well thought out to me.

I'm inclined to use the write side in Synch Rep. So, I'll consider it carefully.

> Again,
> the buffering pqcomm does makes a direct query of the underlying state
> seem pretty suspicious.

Sorry, I didn't get your point. Your concern is only the receiving of a query
from the frontend?

I'm not planning to change the existing communication of a query and result
between the backend and frontend. Non-blocking communication is used in
log-shipping between walsender and walreceiver. In order to speed up log-
shipping, the walsender should send the XLOG records and receive the
reply from walreceiver as concurrently as possible. In other words, I'd like to
prevent the pause that the walsender cannot send the outstanding records
during receiving with blocking. That pause might take long because of slow
network.

Regards,

--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center