Re: non-blocking connections in libpq, fix proposal

Bernhard Herzog <bh(at)intevation(dot)de> writes:
> after finding out that libpq apparently doesn't work properly when
> sending long queries ('long' meaning somewhere larger than 8KB), I had a
> look at the sources and also found some mails in the archives where this
> issue had been discussed. The problem appears to be still present in the
> current CVS version.

Indeed, the "non blocking" support in libpq is completely broken IMHO,
and has been since it was put in; but I've not had time to work on it
myself. And the original developer of the feature apparently never
stressed it hard enough to have a problem, so he insisted there was
no problem :-( I think you're the first person to come along and
actually want to do something about it.

> The problem with pqFlush as it stands now is that it returns EOF when an
> error occurs or when not all data could be sent. The latter case is
> clearly not an error for a non-blocking connection but the caller can't
> distringuish it from an error very well.

Right. The real problem with the design is that the return codes of a
lot of routines aren't adequate to distinguish these cases. However,
I'm dubious that pqFlush is the only place where there's a problem;
AFAIR the problem propagates out to quite a lot of places.

> The second part of the fix modifies pqPutBytes to use pqFlushSome
> instead of pqFlush and to either send all the data or if not all data
> can be sent on a non-blocking connection to at least put all data into
> the output buffer, enlarging it if necessary.

I think that was the same approach I had in mind awhile back --- if
pqPutBytes only fails when it can neither send nor queue data, then
that reduces the number of places that need to cope with "can't send
yet" conditions.

> I've also added a new API function PQflushSome which analogously to
> PQflush just calls pqFlushSome. Programs using PQsendQuery should use
> this new function.

You mean "programs using nonblocking send should use this function", no?

I don't like the name "pqFlushSome" --- to me "flush" means "get rid of
all the data or else". Maybe "pqSendSome"? Any other ideas out there?

> Being new to postgresql development I'm not completely sure how to
> proceed from here. Is it OK if I post the patch here?

Patches should go to pgsql-patches, especially if they're more than
a few lines long.

regards, tom lane